You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/08/07 20:01:42 UTC
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1079
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1079/display/redirect>
Changes:
------------------------------------------
[...truncated 49.89 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.50.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/a4/a5/afbac4c77cc031f73fc02ade4abdc9e2eb113d8c5a681c3b52c06c1bf780/grpcio_status-1.57.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.50.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.50.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.50.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.50.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/71/30/deee2ffb94194437c730a1c6230d9310ab5f73926a2549cdab91453616bb/dnspython-2.4.1-py3-none-any.whl.metadata
Using cached dnspython-2.4.1-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.28.0-py3-none-any.whl (185 kB)
Using cached azure_identity-1.14.0b2-py3-none-any.whl (154 kB)
Using cached azure_storage_blob-12.17.0-py3-none-any.whl (388 kB)
Using cached boto3-1.28.20-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.29.0-py2.py3-none-any.whl (2.7 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.20.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.16.1-py2.py3-none-any.whl (176 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.1-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.2-py3-none-any.whl (415 kB)
Using cached joblib-1.3.1-py3-none-any.whl (301 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.6.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (772 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.20-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.1-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.50.0.dev0-py3-none-any.whl size=3257184 sha256=174c27dbb44fb4f1021274bdea7556768166abe2d64df9df835af13dc51d432e
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0rc3
Uninstalling protobuf-4.24.0rc3:
Successfully uninstalled protobuf-4.24.0rc3
Successfully installed PyJWT-2.8.0 apache-beam-2.50.0.dev0 attrs-23.1.0 azure-core-1.28.0 azure-identity-1.14.0b2 azure-storage-blob-12.17.0 boto3-1.28.20 botocore-1.31.20 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.1 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.29.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.20.0 google-cloud-core-2.3.3 google-cloud-datastore-2.16.1 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.1 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0rc1 hdfs-2.7.0 httplib2-0.22.0 hypothesis-6.82.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.1 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.3 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.6.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807125355.1691420863.035554/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807125355.1691420863.035554/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807125355.1691420863.035554/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807125355.1691420863.035554/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230807150743036466-5997'
createTime: '2023-08-07T15:07:44.002455Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-07_08_07_43-10229001806204194739'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0807125355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-07T15:07:44.002455Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-07_08_07_43-10229001806204194739]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-07_08_07_43-10229001806204194739
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-07_08_07_43-10229001806204194739?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-07_08_07_43-10229001806204194739 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T15:07:54.727Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T15:07:57.161Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T15:07:57.488Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-07_08_07_43-10229001806204194739 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T15:08:13.383Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-07_08_07_43-10229001806204194739 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T20:00:42.648Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-07_08_07_43-10229001806204194739.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T20:00:42.826Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-07T20:00:42.883Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 758, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-07_08_07_43-10229001806204194739?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/synni6by5advy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_LoadTests_Python_Combine_Dataflow_Streaming #1117
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1117/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1116
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1116/display/redirect>
Changes:
------------------------------------------
[...truncated 53.12 KB...]
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/64/b9/416bd3547aab7c2a2f557d281aab30e3f9bf5a695318f6961b9e8eeb0bc3/greenlet-3.0.0rc3-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0rc3-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/0b/50/49e0d7342e5d441d43b525d6c84656ea40aea3e58d530004d07b22bc9b04/websocket_client-1.6.3-py3-none-any.whl.metadata
Using cached websocket_client-1.6.3-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.0-py3-none-any.whl (392 kB)
Using cached boto3-1.28.46-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0rc1-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.0-py2.py3-none-any.whl (181 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.3-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.46-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc3-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0-py2.py3-none-any.whl (91 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.3-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3312222 sha256=ccac010c1a5a12cd32b3ac2b0f9ea4996077362256e9b2466f1ce4986bcbc266
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.0 boto3-1.28.46 botocore-1.31.46 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0rc1 google-apitools-0.5.31 google-auth-2.23.0 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc3 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.7 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.3 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230912
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230912" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0913125355.1694617681.608152/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0913125355.1694617681.608152/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0913125355.1694617681.608152/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0913125355.1694617681.608152/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230913150801609165-4723'
createTime: '2023-09-13T15:08:02.602428Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-13_08_08_02-6299722890580030042'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0913125355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-13T15:08:02.602428Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-13_08_08_02-6299722890580030042]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-13_08_08_02-6299722890580030042
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-13_08_08_02-6299722890580030042?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-13_08_08_02-6299722890580030042 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T15:08:08.083Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T15:08:15.136Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T15:08:15.476Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-13_08_08_02-6299722890580030042 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T15:08:36.770Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T20:00:37.587Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-13_08_08_02-6299722890580030042.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-13_08_08_02-6299722890580030042 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T20:00:37.692Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-13T20:00:37.742Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-13_08_08_02-6299722890580030042?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 8s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
Publishing failed.
The response from https://ge.apache.org/scans/publish/gradle/3.13.2/token was not from Gradle Enterprise.
The specified server address may be incorrect, or your network environment may be interfering.
Please report this problem to your Gradle Enterprise administrator via https://ge.apache.org/help and include the following via copy/paste:
----------
Gradle version: 7.6.2
Plugin version: 3.13.2
Request URL: https://ge.apache.org/scans/publish/gradle/3.13.2/token
Request ID: ca5454d3-6c70-4f1b-9c88-e2b51b2bd7f4
Response status code: 502
Response content type: text/html
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1115
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1115/display/redirect>
Changes:
------------------------------------------
[...truncated 52.53 KB...]
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/d7/71/1620154af1d196a32072b788e50dddeebdd4684235ec5b245ce555989afc/greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/0b/50/49e0d7342e5d441d43b525d6c84656ea40aea3e58d530004d07b22bc9b04/websocket_client-1.6.3-py3-none-any.whl.metadata
Using cached websocket_client-1.6.3-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Downloading azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.1/162.1 kB 1.7 MB/s eta 0:00:00
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.45-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0rc1-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.0-py2.py3-none-any.whl (181 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.3-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.45-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (611 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.3-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3310028 sha256=362794f997748dd31630ef2ed0a9b1c84173842528e7e0dd644a82e57df75ee9
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.3
Uninstalling protobuf-4.24.3:
Successfully uninstalled protobuf-4.24.3
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.0b1 boto3-1.28.45 botocore-1.31.45 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0rc1 google-apitools-0.5.31 google-auth-2.23.0 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc2 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.3 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0912125411.1694531276.682651/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0912125411.1694531276.682651/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0912125411.1694531276.682651/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0912125411.1694531276.682651/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230912150756683648-5001'
createTime: '2023-09-12T15:07:57.730317Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-12_08_07_57-687697229327874203'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0912125411'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-12T15:07:57.730317Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-12_08_07_57-687697229327874203]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-12_08_07_57-687697229327874203
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-12_08_07_57-687697229327874203?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-12_08_07_57-687697229327874203 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T15:08:03.829Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T15:08:05.769Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T15:08:06.089Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-12_08_07_57-687697229327874203 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T15:08:41.931Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
WARNING:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T19:58:36.125Z: JOB_MESSAGE_WARNING: Autoscaling: Unable to reach resize target in zone us-central1-b. ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: Instance 'load-tests-python-dataflo-09120807-1w0f-harness-h2zj' creation failed: The zone 'projects/apache-beam-testing/zones/us-central1-b' does not have enough resources available to fulfill the request. '(resource type:compute)'.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-12_08_07_57-687697229327874203 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T20:24:29.351Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-12_08_07_57-687697229327874203.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T20:24:29.501Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-12T20:24:29.555Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-12_08_07_57-687697229327874203?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 18m 46s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/lf3va2ogwevxq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1114
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1114/display/redirect>
Changes:
------------------------------------------
[...truncated 51.89 KB...]
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/d7/71/1620154af1d196a32072b788e50dddeebdd4684235ec5b245ce555989afc/greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/0b/50/49e0d7342e5d441d43b525d6c84656ea40aea3e58d530004d07b22bc9b04/websocket_client-1.6.3-py3-none-any.whl.metadata
Using cached websocket_client-1.6.3-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.44-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Downloading google_api_core-2.12.0rc1-py3-none-any.whl (121 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.5/121.5 kB 3.9 MB/s eta 0:00:00
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.3-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.44-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (611 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.3-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3309917 sha256=02aa06f3b6072e1b6223a82fd19f719c70e4d06aa2fdd30a1ba4fdd61a08f7b1
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.3
Uninstalling protobuf-4.24.3:
Successfully uninstalled protobuf-4.24.3
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.44 botocore-1.31.44 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0rc1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc2 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.3 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0911125343.1694444885.389798/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0911125343.1694444885.389798/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0911125343.1694444885.389798/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0911125343.1694444885.389798/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230911150805390801-4590'
createTime: '2023-09-11T15:08:06.453787Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-11_08_08_05-16896172010137766831'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0911125343'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-11T15:08:06.453787Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-11_08_08_05-16896172010137766831]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-11_08_08_05-16896172010137766831
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-11_08_08_05-16896172010137766831?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-11_08_08_05-16896172010137766831 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T15:08:17.634Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T15:08:24.467Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T15:08:24.893Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-11_08_08_05-16896172010137766831 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T15:08:35.235Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-11_08_08_05-16896172010137766831 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T20:01:34.767Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-11_08_08_05-16896172010137766831.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T20:01:34.973Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-11T20:01:35.023Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-11_08_08_05-16896172010137766831?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 39s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
https://ge.apache.org/s/fsbdhm5dylcua
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1113
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1113/display/redirect>
Changes:
------------------------------------------
[...truncated 51.84 KB...]
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/d7/71/1620154af1d196a32072b788e50dddeebdd4684235ec5b245ce555989afc/greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.44-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.2-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.44-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (611 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3309917 sha256=a741d1c363991ed1f547ca61f8c94e861579e4e45855e741ba867ab8a32a91a0
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.3
Uninstalling protobuf-4.24.3:
Successfully uninstalled protobuf-4.24.3
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.44 botocore-1.31.44 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc2 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0910125341.1694358473.303006/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0910125341.1694358473.303006/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0910125341.1694358473.303006/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0910125341.1694358473.303006/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230910150753304073-6631'
createTime: '2023-09-10T15:07:54.421746Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-10_08_07_53-15937158436987760'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0910125341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-10T15:07:54.421746Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-10_08_07_53-15937158436987760]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-10_08_07_53-15937158436987760
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-10_08_07_53-15937158436987760?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-10_08_07_53-15937158436987760 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:08:04.816Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:08:06.852Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:08:07.170Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-10_08_07_53-15937158436987760 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:08:16.334Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:13:11.681Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T15:13:25.375Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-10_08_07_53-15937158436987760 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T20:00:43.826Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-10_08_07_53-15937158436987760.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T20:00:43.963Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-10T20:00:44.009Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-10_08_07_53-15937158436987760?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 17s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/o6kgwcebktjfi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1112
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1112/display/redirect>
Changes:
------------------------------------------
[...truncated 51.67 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/d7/71/1620154af1d196a32072b788e50dddeebdd4684235ec5b245ce555989afc/greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.44-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.2-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.44-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc2-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (611 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3309917 sha256=ef407b1cc0d8743b45882014dc7786deb4e9d636489e36c5a82b335729f67ddc
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.3
Uninstalling protobuf-4.24.3:
Successfully uninstalled protobuf-4.24.3
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.44 botocore-1.31.44 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc2 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0909125401.1694272099.761282/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0909125401.1694272099.761282/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0909125401.1694272099.761282/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0909125401.1694272099.761282/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230909150819762212-8616'
createTime: '2023-09-09T15:08:20.991864Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-09_08_08_20-16749366383147749878'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0909125401'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-09T15:08:20.991864Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-09_08_08_20-16749366383147749878]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-09_08_08_20-16749366383147749878
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-09_08_08_20-16749366383147749878?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-09_08_08_20-16749366383147749878 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T15:08:27.435Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T15:08:36.013Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T15:08:37.189Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T15:08:37.415Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-09_08_08_20-16749366383147749878 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T20:00:39.676Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-09_08_08_20-16749366383147749878.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T20:00:39.778Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-09T20:00:39.819Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-09_08_08_20-16749366383147749878 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-09_08_08_20-16749366383147749878?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 9s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/6vktlt2dcspz4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1111
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1111/display/redirect>
Changes:
------------------------------------------
[...truncated 51.68 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/f6/48/2bcf11bc2df159564eac099ea38d80663d291a56fa2f2f561d08bf083dfa/grpcio_status-1.58.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.43-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.2-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.43-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3303059 sha256=94d6a45973f22196dbc6ae92316db6d03379a75719b5c18e3427748b0cff4257
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.3
Uninstalling protobuf-4.24.3:
Successfully uninstalled protobuf-4.24.3
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.43 botocore-1.31.43 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.3 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0908125350.1694185699.325632/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0908125350.1694185699.325632/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0908125350.1694185699.325632/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0908125350.1694185699.325632/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230908150819326555-7992'
createTime: '2023-09-08T15:08:20.268156Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-08_08_08_19-754038974611017331'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0908125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-08T15:08:20.268156Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-08_08_08_19-754038974611017331]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-08_08_08_19-754038974611017331
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-08_08_08_19-754038974611017331?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-08_08_08_19-754038974611017331 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T15:08:25.283Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T15:08:27.574Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T15:08:27.846Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-08_08_08_19-754038974611017331 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T15:08:34.634Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-08_08_08_19-754038974611017331 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T20:00:59.791Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-08_08_08_19-754038974611017331.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T20:00:59.920Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-08T20:00:59.960Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-08_08_08_19-754038974611017331?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/fpt7mu5iz7cas
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1110
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1110/display/redirect>
Changes:
------------------------------------------
[...truncated 51.71 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.42-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.2-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.1-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.42-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3300692 sha256=085da3b530d45ff58d908e3e6225d113d907cb048d68cbb5445f153d92775558
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.42 botocore-1.31.42 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.2 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0907125401.1694099351.179515/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0907125401.1694099351.179515/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0907125401.1694099351.179515/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0907125401.1694099351.179515/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230907150911181062-8023'
createTime: '2023-09-07T15:09:12.315032Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-07_08_09_11-6915908769714256045'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0907125401'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-07T15:09:12.315032Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-07_08_09_11-6915908769714256045]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-07_08_09_11-6915908769714256045
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-07_08_09_11-6915908769714256045?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-07_08_09_11-6915908769714256045 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T15:09:17.598Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T15:09:19.458Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T15:09:19.738Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-07_08_09_11-6915908769714256045 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T15:09:32.033Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-07_08_09_11-6915908769714256045 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T20:14:10.519Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-07_08_09_11-6915908769714256045.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T20:14:10.637Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-07T20:14:10.688Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-07_08_09_11-6915908769714256045?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 8m 34s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/bf5jpzkyshdmu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1109
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1109/display/redirect>
Changes:
------------------------------------------
[...truncated 51.57 KB...]
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.41-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Downloading google_cloud_aiplatform-1.32.0-py2.py3-none-any.whl (2.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 36.4 MB/s eta 0:00:00
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.2-py3-none-any.whl (419 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.1-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.41-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3300033 sha256=e1531e0c890a5fe1cd0318fdad89bfa1ac946b99bf2e62c981412880e344a003
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.41 botocore-1.31.41 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.32.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.2 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0906125350.1694012867.957135/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0906125350.1694012867.957135/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0906125350.1694012867.957135/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0906125350.1694012867.957135/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230906150747958080-4118'
createTime: '2023-09-06T15:07:48.966169Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-06_08_07_48-11075347368887949142'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0906125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-06T15:07:48.966169Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-06_08_07_48-11075347368887949142]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-06_08_07_48-11075347368887949142
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-06_08_07_48-11075347368887949142?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-06_08_07_48-11075347368887949142 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T15:07:54.432Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T15:07:56.456Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T15:07:56.741Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-06_08_07_48-11075347368887949142 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T15:08:13.561Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-06_08_07_48-11075347368887949142 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T20:16:15.551Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-06_08_07_48-11075347368887949142.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T20:16:15.648Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-06T20:16:15.683Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-06_08_07_48-11075347368887949142?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 10m 43s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/z4kbrra3cd2ns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1108
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1108/display/redirect>
Changes:
------------------------------------------
[...truncated 51.51 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.40-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.0-py3-none-any.whl (417 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.1-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.40-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3299113 sha256=eb350c46e9c3f4ea3fd3cbf6ab9b640cc3fa076331114d8b2cd50afe7c5de050
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.40 botocore-1.31.40 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0905125348.1693926653.974428/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0905125348.1693926653.974428/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0905125348.1693926653.974428/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0905125348.1693926653.974428/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230905151053976268-6544'
createTime: '2023-09-05T15:10:55.279730Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-05_08_10_54-1371352388866758541'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0905125348'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-05T15:10:55.279730Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-05_08_10_54-1371352388866758541]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-05_08_10_54-1371352388866758541
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-05_08_10_54-1371352388866758541?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-05_08_10_54-1371352388866758541 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T15:11:03.488Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T15:11:06.028Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T15:11:06.344Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-05_08_10_54-1371352388866758541 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T15:11:20.471Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-05_08_10_54-1371352388866758541 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T20:48:18.581Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-05_08_10_54-1371352388866758541.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T20:48:18.684Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-05T20:48:18.727Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-05_08_10_54-1371352388866758541?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 40m 45s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/zfnkz3ap3gg74
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1107
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1107/display/redirect>
Changes:
------------------------------------------
[...truncated 51.08 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.40-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.84.0-py3-none-any.whl (417 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.1-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.40-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3299113 sha256=70554de2d76fb7d893034e63ed8778da8615f5f4cc0b686937378b760fb13195
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.40 botocore-1.31.40 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.84.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0904125338.1693840083.117996/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0904125338.1693840083.117996/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0904125338.1693840083.117996/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0904125338.1693840083.117996/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230904150803119037-1422'
createTime: '2023-09-04T15:08:04.146814Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-04_08_08_03-1011479574545312762'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0904125338'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-04T15:08:04.146814Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-04_08_08_03-1011479574545312762]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-04_08_08_03-1011479574545312762
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-04_08_08_03-1011479574545312762?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-04_08_08_03-1011479574545312762 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T15:08:08.888Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T15:08:21.226Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T15:08:21.536Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T15:08:21.803Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-04_08_08_03-1011479574545312762 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-04_08_08_03-1011479574545312762 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T20:00:47.133Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-04_08_08_03-1011479574545312762.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T20:00:47.236Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-04T20:00:47.273Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-04_08_08_03-1011479574545312762?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/f354xecwmpuba
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1106
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1106/display/redirect>
Changes:
------------------------------------------
[...truncated 51.06 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.40-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.83.0-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.1-py3-none-any.whl (324 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.40-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3299113 sha256=3ad0c31163a7f9db6315735a8b50669a570117e02b1cb245f582f26f6e7ec97f
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.40 botocore-1.31.40 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.83.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.1 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0903125345.1693753676.249528/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0903125345.1693753676.249528/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0903125345.1693753676.249528/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0903125345.1693753676.249528/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230903150756250480-3619'
createTime: '2023-09-03T15:07:57.158474Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-03_08_07_56-1496317002990797103'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0903125345'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-03T15:07:57.158474Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-03_08_07_56-1496317002990797103]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-03_08_07_56-1496317002990797103
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-03_08_07_56-1496317002990797103?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-03_08_07_56-1496317002990797103 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T15:08:09.458Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T15:08:31.163Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T15:08:31.397Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-03_08_07_56-1496317002990797103 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T15:08:47.209Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-03_08_07_56-1496317002990797103 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T20:00:59.634Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-03_08_07_56-1496317002990797103.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T20:00:59.801Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-03T20:00:59.861Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-03_08_07_56-1496317002990797103?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 7s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/altktmmpgq6fg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1105
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1105/display/redirect>
Changes:
------------------------------------------
[...truncated 51.12 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/7a/4d/ee0386754f2a2cf23dfcd60b3101c02c99cd6e9647cac834ab9dd54d832a/greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.40-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.83.0-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.40-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (615 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3299113 sha256=7d4767d2b4c29311d763c0de7a45bfaec08cb65e1eadd65cf7397b301ab2365a
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.40 botocore-1.31.40 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0rc1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.83.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0902125338.1693667264.329539/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0902125338.1693667264.329539/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0902125338.1693667264.329539/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0902125338.1693667264.329539/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230902150744330498-2773'
createTime: '2023-09-02T15:07:45.315639Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-02_08_07_44-8023100543325930444'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0902125338'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-02T15:07:45.315639Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-02_08_07_44-8023100543325930444]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-02_08_07_44-8023100543325930444
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-02_08_07_44-8023100543325930444?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-02_08_07_44-8023100543325930444 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T15:07:50.909Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T15:07:53.068Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T15:07:53.353Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-02_08_07_44-8023100543325930444 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T15:08:19.327Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-02_08_07_44-8023100543325930444 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T20:00:45.539Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-02_08_07_44-8023100543325930444.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T20:00:45.655Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-02T20:00:45.691Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-02_08_07_44-8023100543325930444?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 16s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/ugi23o5ills6i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1104
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1104/display/redirect>
Changes:
------------------------------------------
[...truncated 51.09 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.39-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.83.0-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.39-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3299113 sha256=0892dd0667f71ba12e4083380fafe5ef708db17add6c9aa6f3837dc36b1d344b
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.39 botocore-1.31.39 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.83.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0901125349.1693580888.962600/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0901125349.1693580888.962600/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0901125349.1693580888.962600/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0901125349.1693580888.962600/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230901150808963632-4733'
createTime: '2023-09-01T15:08:10.116972Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-09-01_08_08_09-4639791401496880020'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0901125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-09-01T15:08:10.116972Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-09-01_08_08_09-4639791401496880020]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-09-01_08_08_09-4639791401496880020
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-01_08_08_09-4639791401496880020?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-01_08_08_09-4639791401496880020 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T15:08:15.471Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T15:08:17.520Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T15:08:17.833Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-01_08_08_09-4639791401496880020 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T15:08:39.645Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-09-01_08_08_09-4639791401496880020 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T20:00:55.035Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-09-01_08_08_09-4639791401496880020.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T20:00:55.311Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-09-01T20:00:55.353Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-09-01_08_08_09-4639791401496880020?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 14s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/64ppdxfrw7wog
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1103
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1103/display/redirect>
Changes:
------------------------------------------
[...truncated 52.36 KB...]
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.38-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.7-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.38-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3289929 sha256=e6c56c0cbf3b823a38151c5bf755d4961076d13be6b0841ea2286eadfab115dc
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.38 botocore-1.31.38 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.7 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0831125351.1693494526.908087/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0831125351.1693494526.908087/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0831125351.1693494526.908087/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0831125351.1693494526.908087/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230831150846909169-1453'
createTime: '2023-08-31T15:08:47.826830Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-31_08_08_47-1213101763353863110'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0831125351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-31T15:08:47.826830Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-31_08_08_47-1213101763353863110]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-31_08_08_47-1213101763353863110
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-31_08_08_47-1213101763353863110?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-31_08_08_47-1213101763353863110 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T15:08:53.325Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T15:08:55.585Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T15:08:55.828Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-31_08_08_47-1213101763353863110 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T15:09:14.841Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T18:25:17.627Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(f3cbe7d85813c08b): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(96fee1698668f660): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T20:10:37.627Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-31_08_08_47-1213101763353863110.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T20:10:37.724Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-31T20:10:37.761Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-31_08_08_47-1213101763353863110 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-31_08_08_47-1213101763353863110?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 4m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/ovcx64aapq6y6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1102
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1102/display/redirect>
Changes:
------------------------------------------
[...truncated 51.45 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.37-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.7-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.37-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3286601 sha256=365a1fcfa3962434428a4c2d6b5f6f4378eb5ae3cda4c7bed04aa7d46127c1d3
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.37 botocore-1.31.37 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.7 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0830125355.1693408068.156239/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0830125355.1693408068.156239/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0830125355.1693408068.156239/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0830125355.1693408068.156239/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230830150748157175-7753'
createTime: '2023-08-30T15:07:49.129443Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-30_08_07_48-9134753419741292426'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0830125355'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-30T15:07:49.129443Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-30_08_07_48-9134753419741292426]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-30_08_07_48-9134753419741292426
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-30_08_07_48-9134753419741292426?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-30_08_07_48-9134753419741292426 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T15:07:56.578Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T15:08:04.051Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T15:08:04.306Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-30_08_07_48-9134753419741292426 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T15:08:28.175Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-30_08_07_48-9134753419741292426 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T20:10:35.756Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-30_08_07_48-9134753419741292426.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T20:10:35.886Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-30T20:10:35.938Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-30_08_07_48-9134753419741292426?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 4m 56s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/vhqtvf2chsw7y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1101
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1101/display/redirect>
Changes:
------------------------------------------
[...truncated 51.09 KB...]
Using cached s3transfer-0.6.2-py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.36-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.7-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached botocore-1.31.36-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3282927 sha256=945732816cacd5bd63ac75d3419ca11fb25ea5b053a3c9d7f2dbb4e93d647a73
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.36 botocore-1.31.36 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.7 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0829125353.1693321674.786066/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0829125353.1693321674.786066/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0829125353.1693321674.786066/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0829125353.1693321674.786066/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230829150754787039-9985'
createTime: '2023-08-29T15:07:55.806200Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-29_08_07_55-13409799506349067533'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0829125353'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-29T15:07:55.806200Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-29_08_07_55-13409799506349067533]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-29_08_07_55-13409799506349067533
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-29_08_07_55-13409799506349067533?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-29_08_07_55-13409799506349067533 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T15:08:01.943Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-f.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T15:08:04.109Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T15:08:04.432Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-f...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-29_08_07_55-13409799506349067533 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T15:08:37.377Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-29_08_07_55-13409799506349067533 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T20:01:15.971Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-29_08_07_55-13409799506349067533.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T20:01:16.129Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-29T20:01:16.173Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-29_08_07_55-13409799506349067533?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 10s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/645k7lh2i3jeu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1100
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1100/display/redirect>
Changes:
------------------------------------------
[...truncated 50.45 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.35-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.35-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3269190 sha256=88dd6e245668e4ee6972a8d3684dbd5dc9fd7f22d523870eba111cad58589325
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.35 botocore-1.31.35 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0828125351.1693235293.887050/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0828125351.1693235293.887050/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0828125351.1693235293.887050/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0828125351.1693235293.887050/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230828150813888067-3472'
createTime: '2023-08-28T15:08:15.028121Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-28_08_08_14-15623451970542356506'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0828125351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-28T15:08:15.028121Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-28_08_08_14-15623451970542356506]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-28_08_08_14-15623451970542356506
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-28_08_08_14-15623451970542356506?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-28_08_08_14-15623451970542356506 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T15:08:21.003Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T15:08:22.849Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T15:08:23.111Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-28_08_08_14-15623451970542356506 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T15:08:50.858Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T20:00:44.148Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-28_08_08_14-15623451970542356506.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T20:00:44.274Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-28T20:00:44.316Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-28_08_08_14-15623451970542356506 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-28_08_08_14-15623451970542356506?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 54m 58s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/mmmqpnnhjlvnq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1099
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1099/display/redirect>
Changes:
------------------------------------------
[...truncated 50.43 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.35-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.35-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3269190 sha256=2aa0927c89768366be7c53c5f73e9efe2827954f331c3c606d1eb75e3911d9a1
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.35 botocore-1.31.35 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0827125347.1693148870.963544/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0827125347.1693148870.963544/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0827125347.1693148870.963544/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0827125347.1693148870.963544/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230827150750964518-2206'
createTime: '2023-08-27T15:07:51.976825Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-27_08_07_51-17761939501690002273'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0827125347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-27T15:07:51.976825Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-27_08_07_51-17761939501690002273]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-27_08_07_51-17761939501690002273
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-27_08_07_51-17761939501690002273?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-27_08_07_51-17761939501690002273 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T15:07:56.813Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T15:07:58.963Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T15:07:59.249Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-27_08_07_51-17761939501690002273 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T15:08:29.818Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T20:00:40.631Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-27_08_07_51-17761939501690002273.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T20:00:40.753Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-27T20:00:40.792Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-27_08_07_51-17761939501690002273 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-27_08_07_51-17761939501690002273?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/vpragl5njmquc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1098
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1098/display/redirect>
Changes:
------------------------------------------
[...truncated 50.46 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/2c/ff/e43d23c0ec5c7d30d7293667ac2cc69b9024c3504c0cd8267836c2ce3874/grpcio_status-1.58.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.35-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.35-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.58.0rc1-py3-none-any.whl (14 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3269190 sha256=b1fca4c82f238c8c294fe32a2d95934eded9cbdfb607c497cf5d5c0c3c8f0adf
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.2
Uninstalling protobuf-4.24.2:
Successfully uninstalled protobuf-4.24.2
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.35 botocore-1.31.35 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.58.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0826125344.1693062470.755731/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0826125344.1693062470.755731/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0826125344.1693062470.755731/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0826125344.1693062470.755731/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230826150750756705-8048'
createTime: '2023-08-26T15:07:51.704570Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-26_08_07_51-9800544226793391506'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0826125344'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-26T15:07:51.704570Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-26_08_07_51-9800544226793391506]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-26_08_07_51-9800544226793391506
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-26_08_07_51-9800544226793391506?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-26_08_07_51-9800544226793391506 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T15:07:57.683Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T15:07:59.903Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T15:08:00.171Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-26_08_07_51-9800544226793391506 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T15:08:17.788Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-26_08_07_51-9800544226793391506 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T20:01:15.443Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-26_08_07_51-9800544226793391506.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T20:01:15.564Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-26T20:01:15.612Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-26_08_07_51-9800544226793391506?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 10s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/petr73fetxhms
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1097
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1097/display/redirect>
Changes:
------------------------------------------
[...truncated 50.46 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.34-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.34-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3269153 sha256=1563729b188fad6577693a5442ee7c877bc26fa22f429f7adf8b1dec207b145c
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.34 botocore-1.31.34 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0825125349.1692976066.781166/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0825125349.1692976066.781166/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0825125349.1692976066.781166/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0825125349.1692976066.781166/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230825150746782131-7268'
createTime: '2023-08-25T15:07:47.827923Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-25_08_07_47-4044771558271621633'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0825125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-25T15:07:47.827923Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-25_08_07_47-4044771558271621633]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-25_08_07_47-4044771558271621633
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-25_08_07_47-4044771558271621633?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-25_08_07_47-4044771558271621633 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T15:07:53.982Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T15:07:55.900Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T15:07:56.178Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-25_08_07_47-4044771558271621633 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T15:08:18.182Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-25_08_07_47-4044771558271621633 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T20:00:53.859Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-25_08_07_47-4044771558271621633.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T20:00:54.005Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-25T20:00:54.043Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-25_08_07_47-4044771558271621633?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 15s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/gmcwqspcmxluq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1096
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1096/display/redirect>
Changes:
------------------------------------------
[...truncated 50.49 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.33-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.33-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3265999 sha256=085486de17ed93d26a42157151241a0667136eb9012bb9ca0f5c486ae9375727
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.33 botocore-1.31.33 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0824125349.1692889666.787969/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0824125349.1692889666.787969/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0824125349.1692889666.787969/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0824125349.1692889666.787969/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230824150746788995-5246'
createTime: '2023-08-24T15:07:47.751408Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-24_08_07_47-6003693261015419100'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0824125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-24T15:07:47.751408Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-24_08_07_47-6003693261015419100]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-24_08_07_47-6003693261015419100
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-24_08_07_47-6003693261015419100?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-24_08_07_47-6003693261015419100 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T15:07:53.210Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T15:07:55.328Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T15:07:55.634Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-24_08_07_47-6003693261015419100 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T15:08:04.707Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-24_08_07_47-6003693261015419100 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T20:01:21.603Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-24_08_07_47-6003693261015419100.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T20:01:21.728Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-24T20:01:21.765Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-24_08_07_47-6003693261015419100?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 51s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/wd4wrhl4lgb76
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1095
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1095/display/redirect>
Changes:
------------------------------------------
[...truncated 50.61 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/4b/4a/3176388095e5bae6e6a1fbee66c438809230ae0196e7de4af12c5e75c509/websocket_client-1.6.2-py3-none-any.whl.metadata
Using cached websocket_client-1.6.2-py3-none-any.whl.metadata (7.5 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.3-py3-none-any.whl (191 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.32-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.32-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.2-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3265261 sha256=5e6d47838fd9717451a8e84468fc9cb6628363a62bd1923e6b60fae145c01ad1
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.3 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.32 botocore-1.31.32 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.2 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0823125340.1692803259.958126/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0823125340.1692803259.958126/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0823125340.1692803259.958126/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0823125340.1692803259.958126/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230823150739959041-5687'
createTime: '2023-08-23T15:07:41.042627Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-23_08_07_40-1469081200195283417'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0823125340'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-23T15:07:41.042627Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-23_08_07_40-1469081200195283417]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-23_08_07_40-1469081200195283417
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-23_08_07_40-1469081200195283417?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-23_08_07_40-1469081200195283417 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:07:49.294Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:07:51.569Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:07:51.868Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-23_08_07_40-1469081200195283417 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:08:07.312Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:10:35.843Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T15:11:04.870Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T20:01:12.677Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-23_08_07_40-1469081200195283417.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T20:01:12.852Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-23T20:01:12.901Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-23_08_07_40-1469081200195283417 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-23_08_07_40-1469081200195283417?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 52s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/rzpjhj3yo6nz6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1094
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1094/display/redirect>
Changes:
------------------------------------------
[...truncated 50.50 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.31-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.31-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3264723 sha256=9121aa6c42e79cb3f7baf18835adbb74641556bb308b27151c515cc900957cef
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.31 botocore-1.31.31 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0822125341.1692716875.399629/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0822125341.1692716875.399629/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0822125341.1692716875.399629/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0822125341.1692716875.399629/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230822150755400636-6556'
createTime: '2023-08-22T15:07:56.378914Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-22_08_07_55-9864054959676673042'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0822125341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-22T15:07:56.378914Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-22_08_07_55-9864054959676673042]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-22_08_07_55-9864054959676673042
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-22_08_07_55-9864054959676673042?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-22_08_07_55-9864054959676673042 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T15:08:02.467Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T15:08:04.899Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T15:08:05.193Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-22_08_07_55-9864054959676673042 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T15:08:13.995Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-22_08_07_55-9864054959676673042 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T20:00:53.022Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-22_08_07_55-9864054959676673042.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T20:00:53.129Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-22T20:00:53.182Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-22_08_07_55-9864054959676673042?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 11s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/7k6hszjx6fpis
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1093
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1093/display/redirect?page=changes>
Changes:
[noreply] Bump github.com/aws/aws-sdk-go-v2 from 1.20.1 to 1.20.3 in /sdks
------------------------------------------
[...truncated 51.66 KB...]
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.30-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.31.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.6-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.30-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3262073 sha256=ff6d000fcbec853031d5c429b441677e6404ff53c1ae43e9f78a59c5318b3fee
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.30 botocore-1.31.30 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.31.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.6 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0821125347.1692630469.266875/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0821125347.1692630469.266875/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0821125347.1692630469.266875/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0821125347.1692630469.266875/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230821150749268077-5768'
createTime: '2023-08-21T15:07:50.196771Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-21_08_07_49-14582936197018936362'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0821125347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-21T15:07:50.196771Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-21_08_07_49-14582936197018936362]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-21_08_07_49-14582936197018936362
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-21_08_07_49-14582936197018936362?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-21_08_07_49-14582936197018936362 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T15:07:54.933Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T15:07:56.671Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T15:07:56.956Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-21_08_07_49-14582936197018936362 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T15:08:30.051Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T16:05:46.485Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(b080e4f4f6272a36): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T16:05:55.468Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(b080e4f4f6272a36): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
WARNING:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T19:02:51.673Z: JOB_MESSAGE_WARNING: The ****s of given job are going to be updated.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T20:00:42.663Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-21_08_07_49-14582936197018936362.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T20:00:42.764Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-21T20:00:42.800Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-21_08_07_49-14582936197018936362 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-21_08_07_49-14582936197018936362?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 54m 19s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/p3ngpa55lylwc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1092
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1092/display/redirect>
Changes:
------------------------------------------
[...truncated 50.48 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.30-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.5-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.30-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3262073 sha256=4e948ed1ba95dafa0a957c122742b3d756bf53f57658d6293cdc7e8dbb921c08
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.30 botocore-1.31.30 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.5 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0820125344.1692544072.989480/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0820125344.1692544072.989480/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0820125344.1692544072.989480/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0820125344.1692544072.989480/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230820150752990471-9450'
createTime: '2023-08-20T15:07:53.865042Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-20_08_07_53-15552152414980192050'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0820125344'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-20T15:07:53.865042Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-20_08_07_53-15552152414980192050]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-20_08_07_53-15552152414980192050
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-20_08_07_53-15552152414980192050?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-20_08_07_53-15552152414980192050 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T15:08:01.332Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T15:08:03.527Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T15:08:03.912Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-20_08_07_53-15552152414980192050 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T15:08:34.698Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-20_08_07_53-15552152414980192050 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T20:00:40.798Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-20_08_07_53-15552152414980192050.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T20:00:40.941Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-20T20:00:40.979Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-20_08_07_53-15552152414980192050?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 11s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/me6bxk5jseu4y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1091
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1091/display/redirect>
Changes:
------------------------------------------
[...truncated 50.51 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.30-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.3-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.5-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.30-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3262076 sha256=e9d1db8fc38c08c0defb9e562ca140519c49ec9f9be752a920a6fd1bf0cd4833
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.1
Uninstalling protobuf-4.24.1:
Successfully uninstalled protobuf-4.24.1
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.30 botocore-1.31.30 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.3 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.5 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0819125335.1692457667.158374/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0819125335.1692457667.158374/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0819125335.1692457667.158374/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0819125335.1692457667.158374/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230819150747159331-9147'
createTime: '2023-08-19T15:07:48.041564Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-19_08_07_47-8784854442315646440'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0819125335'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-19T15:07:48.041564Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-19_08_07_47-8784854442315646440]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-19_08_07_47-8784854442315646440
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-19_08_07_47-8784854442315646440?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-19_08_07_47-8784854442315646440 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T15:07:54.332Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T15:07:56.416Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T15:07:56.680Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-19_08_07_47-8784854442315646440 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T15:08:09.665Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T20:00:37.534Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-19_08_07_47-8784854442315646440.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T20:00:37.614Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-19T20:00:37.652Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-19_08_07_47-8784854442315646440 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-19_08_07_47-8784854442315646440?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 10s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/qszuanjvf3hu6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1090
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1090/display/redirect>
Changes:
------------------------------------------
[...truncated 52.51 KB...]
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.29-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.5-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.29-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3261151 sha256=32933b7a42f62b6103d6d71c3d952c748a8cb89e7dfe43fc2862c5e2fd931a81
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.29 botocore-1.31.29 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.5 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0818125350.1692371278.539134/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0818125350.1692371278.539134/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0818125350.1692371278.539134/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0818125350.1692371278.539134/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230818150758540380-8829'
createTime: '2023-08-18T15:07:59.579825Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-18_08_07_59-179842436548622322'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0818125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-18T15:07:59.579825Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-18_08_07_59-179842436548622322]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-18_08_07_59-179842436548622322
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-18_08_07_59-179842436548622322?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-18_08_07_59-179842436548622322 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-18T15:08:05.471Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-18T15:08:07.747Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-18T15:08:08.135Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-18_08_07_59-179842436548622322 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-18T15:08:37.794Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-08-18T19:17:28.604Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(23547a800c38848d): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-18_08_07_59-179842436548622322 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-18_08_07_59-179842436548622322?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 54m 20s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
Publishing failed.
The response from https://ge.apache.org/scans/publish/gradle/3.13.2/token was not from Gradle Enterprise.
The specified server address may be incorrect, or your network environment may be interfering.
Please report this problem to your Gradle Enterprise administrator via https://ge.apache.org/help and include the following via copy/paste:
----------
Gradle version: 7.6.2
Plugin version: 3.13.2
Request URL: https://ge.apache.org/scans/publish/gradle/3.13.2/token
Request ID: 39f11bff-3d27-46a5-a48b-2bf7d1f95d0f
Response status code: 502
Response content type: text/html
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1089
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1089/display/redirect>
Changes:
------------------------------------------
[...truncated 51.96 KB...]
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.28-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev1-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.4-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.28-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3261155 sha256=98ca025f566e96a8cc458e601d2c25de7adc9263f14a9313e71093be9fb42691
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.28 botocore-1.31.28 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev1 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.1 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.5 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0817125351.1692284874.409943/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0817125351.1692284874.409943/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0817125351.1692284874.409943/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0817125351.1692284874.409943/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230817150754410910-1825'
createTime: '2023-08-17T15:07:55.403131Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-17_08_07_54-11055652507357529805'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0817125351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-17T15:07:55.403131Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-17_08_07_54-11055652507357529805]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-17_08_07_54-11055652507357529805
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-17_08_07_54-11055652507357529805?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-17_08_07_54-11055652507357529805 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T15:07:58.935Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T15:08:01.492Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T15:08:01.750Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-17_08_07_54-11055652507357529805 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T15:08:21.762Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-17_08_07_54-11055652507357529805 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T20:00:51.977Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-17_08_07_54-11055652507357529805.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-17T20:00:52.108Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-17_08_07_54-11055652507357529805?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 15s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
Publishing failed.
The response from https://ge.apache.org/scans/publish/gradle/3.13.2/token was not from Gradle Enterprise.
The specified server address may be incorrect, or your network environment may be interfering.
Please report this problem to your Gradle Enterprise administrator via https://ge.apache.org/help and include the following via copy/paste:
----------
Gradle version: 7.6.2
Plugin version: 3.13.2
Request URL: https://ge.apache.org/scans/publish/gradle/3.13.2/token
Request ID: b0ba007d-6a2c-47f6-9c17-b7c5f8041931
Response status code: 502
Response content type: text/html
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1088
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1088/display/redirect>
Changes:
------------------------------------------
[...truncated 50.40 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.27-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.4-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.27-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.6.2-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3261158 sha256=0198b17731b5f6744dfb09dcd8d7d25b9d0223626e62dc24b9ac8c8c4da5026b
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.27 botocore-1.31.27 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.2 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0816125351.1692198471.224474/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0816125351.1692198471.224474/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0816125351.1692198471.224474/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0816125351.1692198471.224474/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230816150751225434-1087'
createTime: '2023-08-16T15:07:54.516286Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-16_08_07_51-13026894485037550381'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0816125351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-16T15:07:54.516286Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-16_08_07_51-13026894485037550381]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-16_08_07_51-13026894485037550381
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-16_08_07_51-13026894485037550381?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-16_08_07_51-13026894485037550381 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T15:07:59.233Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T15:08:01.221Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T15:08:01.547Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-16_08_07_51-13026894485037550381 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T15:08:26.950Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-16_08_07_51-13026894485037550381 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T20:00:40.192Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-16_08_07_51-13026894485037550381.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T20:00:40.301Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-16T20:00:40.344Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-16_08_07_51-13026894485037550381?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 9s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/2gbvb2iczm6jg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1087
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1087/display/redirect>
Changes:
------------------------------------------
[...truncated 50.08 KB...]
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.2-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.26-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.0-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.4-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.26-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3260562 sha256=78328d337f31b5d097e0a96faf8aebab1602092ff9c1bfb437944a3e3f720523
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.2 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.26 botocore-1.31.26 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.11.0 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0815134947.1692112070.530277/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0815134947.1692112070.530277/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0815134947.1692112070.530277/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0815134947.1692112070.530277/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230815150750531246-8729'
createTime: '2023-08-15T15:07:54.128537Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-15_08_07_50-11416184213631712351'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0815134947'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-15T15:07:54.128537Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-15_08_07_50-11416184213631712351]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-15_08_07_50-11416184213631712351
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-15_08_07_50-11416184213631712351?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-15_08_07_50-11416184213631712351 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T15:07:58.445Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T15:08:00.355Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T15:08:00.666Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-15_08_07_50-11416184213631712351 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T15:08:34.054Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T20:00:41.898Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-15_08_07_50-11416184213631712351.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T20:00:42.015Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-15T20:00:42.068Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-15_08_07_50-11416184213631712351 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-15_08_07_50-11416184213631712351?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 17s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/e3pjlf6emop6u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1086
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1086/display/redirect?page=changes>
Changes:
[noreply] Add Vertex AI image classification integration test (#27958)
------------------------------------------
[...truncated 50.08 KB...]
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.1-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.25-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.4-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.25-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3260568 sha256=a4ff6f9f6743ec9744216e157644788dc538100129ed9b39471350c2c3d68657
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.1 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.25 botocore-1.31.25 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0814125343.1692025690.971701/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0814125343.1692025690.971701/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0814125343.1692025690.971701/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0814125343.1692025690.971701/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230814150810972693-6423'
createTime: '2023-08-14T15:08:12.040134Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-14_08_08_11-12935301229472539894'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0814125343'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-14T15:08:12.040134Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-14_08_08_11-12935301229472539894]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-14_08_08_11-12935301229472539894
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-14_08_08_11-12935301229472539894?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-14_08_08_11-12935301229472539894 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T15:08:19.566Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T15:08:28.145Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T15:08:28.430Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-14_08_08_11-12935301229472539894 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T15:08:47.625Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-14_08_08_11-12935301229472539894 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T20:00:43.821Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-14_08_08_11-12935301229472539894.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T20:00:43.926Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-14T20:00:43.981Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-14_08_08_11-12935301229472539894?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 6s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/7flqnvtu6tquq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1085
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1085/display/redirect>
Changes:
------------------------------------------
[...truncated 49.80 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.1-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.25-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.4-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.25-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3259086 sha256=cd8c44b127c35c913eafd6fb8b7c7bb9b5c2ede4777b36960905599a4486ebfc
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.1 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.25 botocore-1.31.25 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0813125349.1691939269.690531/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0813125349.1691939269.690531/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0813125349.1691939269.690531/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0813125349.1691939269.690531/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230813150749691504-3543'
createTime: '2023-08-13T15:07:50.686032Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-13_08_07_50-4827225711920960570'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0813125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-13T15:07:50.686032Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-13_08_07_50-4827225711920960570]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-13_08_07_50-4827225711920960570
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-13_08_07_50-4827225711920960570?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-13_08_07_50-4827225711920960570 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T15:07:54.802Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T15:07:56.896Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T15:07:57.166Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-13_08_07_50-4827225711920960570 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T15:08:26.820Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-13_08_07_50-4827225711920960570 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T20:00:58.951Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-13_08_07_50-4827225711920960570.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T20:00:59.085Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-13T20:00:59.125Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-13_08_07_50-4827225711920960570?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 8s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/h5y73wl7unf3g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1084
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1084/display/redirect>
Changes:
------------------------------------------
[...truncated 49.83 KB...]
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.1-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.25-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.1-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.3-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.25-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3259086 sha256=c52b2ea117fda96d2abb015f771be158395dead0f80a4b81d53f673cc884bf9c
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.1 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.25 botocore-1.31.25 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.1 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0812125350.1691852869.070134/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0812125350.1691852869.070134/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0812125350.1691852869.070134/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0812125350.1691852869.070134/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230812150749071082-6029'
createTime: '2023-08-12T15:07:49.989657Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-12_08_07_49-7768979073221823935'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0812125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-12T15:07:49.989657Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-12_08_07_49-7768979073221823935]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-12_08_07_49-7768979073221823935
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-12_08_07_49-7768979073221823935?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-12_08_07_49-7768979073221823935 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T15:07:55.290Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T15:07:56.999Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T15:07:57.295Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-12_08_07_49-7768979073221823935 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T15:08:06.054Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-12_08_07_49-7768979073221823935 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T20:00:40.625Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-12_08_07_49-7768979073221823935.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T20:00:40.716Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-12T20:00:40.766Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-12_08_07_49-7768979073221823935?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 14s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/vxkvo6atxusdg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1083
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1083/display/redirect>
Changes:
------------------------------------------
[...truncated 50.02 KB...]
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0 (from boto3<2,>=1.9->apache-beam==2.51.0.dev0)
Using cached s3transfer-0.6.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0) (23.1)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.51.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.51.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/d0/3f/347d93056572fdbd64d4f0fc58a18d420763a7118f8b177437d9dab0ae6f/grpcio_status-1.57.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.51.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.51.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.51.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.51.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.51.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.51.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.51.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.51.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.51.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.51.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.1-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.24-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.30.0-py2.py3-none-any.whl (2.8 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.3-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.24-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.51.0.dev0-py3-none-any.whl size=3225946 sha256=2b88dc798b97a9316ad0f82f805ec0340f24c43ea2623f0b2b6fd39a8f6cab01
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.51.0.dev0 attrs-23.1.0 azure-core-1.29.1 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.24 botocore-1.31.24 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.30.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0811125341.1691766515.277184/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0811125341.1691766515.277184/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0811125341.1691766515.277184/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0811125341.1691766515.277184/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230811150835278142-8243'
createTime: '2023-08-11T15:08:36.303697Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-11_08_08_35-15723274632313268860'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0811125341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-11T15:08:36.303697Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-11_08_08_35-15723274632313268860]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-11_08_08_35-15723274632313268860
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-11_08_08_35-15723274632313268860?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-11_08_08_35-15723274632313268860 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T15:08:41.421Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T15:08:43.479Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T15:08:43.728Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-11_08_08_35-15723274632313268860 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T15:09:03.269Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-11_08_08_35-15723274632313268860 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T20:00:41.619Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-11_08_08_35-15723274632313268860.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T20:00:41.755Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-11T20:00:41.797Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-11_08_08_35-15723274632313268860?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 10s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/xu7jq6h3qskjs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1082
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1082/display/redirect>
Changes:
------------------------------------------
[...truncated 50.06 KB...]
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0 (from boto3<2,>=1.9->apache-beam==2.50.0.dev0)
Using cached s3transfer-0.6.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0) (23.1)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.50.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/a4/a5/afbac4c77cc031f73fc02ade4abdc9e2eb113d8c5a681c3b52c06c1bf780/grpcio_status-1.57.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.50.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.50.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.50.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.50.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.1-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.23-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.29.0-py2.py3-none-any.whl (2.7 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.3-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.23-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.50.0.dev0-py3-none-any.whl size=3225580 sha256=5d041ed6a86dbd615941aaa3c5c699e2f87de3198e6c536feba3906e3398294c
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.50.0.dev0 attrs-23.1.0 azure-core-1.29.1 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.23 botocore-1.31.23 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.29.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0rc1 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.82.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0810125347.1691680075.193480/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0810125347.1691680075.193480/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0810125347.1691680075.193480/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0810125347.1691680075.193480/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230810150755194526-6969'
createTime: '2023-08-10T15:07:56.339826Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-10_08_07_55-12157205673782733569'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0810125347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-10T15:07:56.339826Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-10_08_07_55-12157205673782733569]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-10_08_07_55-12157205673782733569
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-10_08_07_55-12157205673782733569?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-10_08_07_55-12157205673782733569 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T15:08:01.749Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T15:08:03.965Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T15:08:04.200Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-10_08_07_55-12157205673782733569 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T15:08:15.137Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-10_08_07_55-12157205673782733569 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T20:00:57.584Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-10_08_07_55-12157205673782733569.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T20:00:57.682Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-10T20:00:57.729Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 757, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-10_08_07_55-12157205673782733569?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 11s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/73cyskmbnejco
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1081
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1081/display/redirect>
Changes:
------------------------------------------
[...truncated 50.06 KB...]
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0 (from boto3<2,>=1.9->apache-beam==2.50.0.dev0)
Using cached s3transfer-0.6.1-py3-none-any.whl (79 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0) (23.1)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.50.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/a4/a5/afbac4c77cc031f73fc02ade4abdc9e2eb113d8c5a681c3b52c06c1bf780/grpcio_status-1.57.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.50.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.50.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.50.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.50.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.29.0-py3-none-any.whl (188 kB)
Using cached azure_identity-1.14.0-py3-none-any.whl (160 kB)
Using cached azure_storage_blob-12.18.0b1-py3-none-any.whl (392 kB)
Using cached boto3-1.28.22-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.29.0-py2.py3-none-any.whl (2.7 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.17.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.3-py3-none-any.whl (415 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.22-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.50.0.dev0-py3-none-any.whl size=3226006 sha256=d99681157aaaa6e86f236ced46dda44d9350bc415646238f584e0d49030239d5
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0
Uninstalling protobuf-4.24.0:
Successfully uninstalled protobuf-4.24.0
Successfully installed PyJWT-2.8.0 apache-beam-2.50.0.dev0 attrs-23.1.0 azure-core-1.29.0 azure-identity-1.14.0 azure-storage-blob-12.18.0b1 boto3-1.28.22 botocore-1.31.22 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.29.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.17.0 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0rc1 hdfs-2.7.1 httplib2-0.22.0 hypothesis-6.82.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.8.8 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809125349.1691593673.273973/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809125349.1691593673.273973/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809125349.1691593673.273973/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809125349.1691593673.273973/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230809150753274974-5825'
createTime: '2023-08-09T15:07:54.457315Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-09_08_07_53-10829342549033890404'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0809125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-09T15:07:54.457315Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-09_08_07_53-10829342549033890404]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-09_08_07_53-10829342549033890404
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-09_08_07_53-10829342549033890404?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-09_08_07_53-10829342549033890404 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T15:07:58.493Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T15:08:00.318Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T15:08:00.631Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-09_08_07_53-10829342549033890404 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T15:08:23.401Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-09_08_07_53-10829342549033890404 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T20:00:46.745Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-09_08_07_53-10829342549033890404.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T20:00:46.832Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-09T20:00:46.871Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 758, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-09_08_07_53-10829342549033890404?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/7yvyqpd4ufw52
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1080
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1080/display/redirect>
Changes:
------------------------------------------
[...truncated 50.02 KB...]
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for googleapis-common-protos<2.0.dev0,>=1.56.2 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.18.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0) (23.1)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-storage<3.0.0dev,>=1.32.0 from https://files.pythonhosted.org/packages/88/14/c9d4faae7ea4bff4405152cbc762b61100aa6949273b4eb3203d23308670/google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl.metadata (6.0 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for google-cloud-resource-manager<3.0.0dev,>=1.3.3 from https://files.pythonhosted.org/packages/d2/f3/e7d1848edf2f4714d8925e6c749464c190e696b3c85264441c2b3787ae72/google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.50.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_resumable_media-2.5.0-py2.py3-none-any.whl (77 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.50.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/a4/a5/afbac4c77cc031f73fc02ade4abdc9e2eb113d8c5a681c3b52c06c1bf780/grpcio_status-1.57.0rc1-py3-none-any.whl.metadata
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.50.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.50.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.50.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.50.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/fe/17/f43b7c9ccf399d72038042ee72785c305f6c6fdc6231942f8ab99d995742/exceptiongroup-1.1.2-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.2-py3-none-any.whl.metadata (6.1 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/71/30/deee2ffb94194437c730a1c6230d9310ab5f73926a2549cdab91453616bb/dnspython-2.4.1-py3-none-any.whl.metadata
Using cached dnspython-2.4.1-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (1.2.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.50.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/cb/e7/5e43745003bf1f90668c7be23fc5952b3a2b9c2558f16749411c18039b36/charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/c7/be/6f1924d468b309dd7c3e33c8665286f5ff92ac8ad24fbfa7933f9ddb207e/greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached wrapt-1.15.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.50.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.50.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/d3/a3/63e9329c8cc9be6153e919e17d0ef5b60d537fed78564872951b95bcc17c/websocket_client-1.6.1-py3-none-any.whl.metadata
Using cached websocket_client-1.6.1-py3-none-any.whl.metadata (7.6 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.50.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.50.0.dev0)
Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.50.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Using cached azure_core-1.28.0-py3-none-any.whl (185 kB)
Using cached azure_identity-1.14.0b2-py3-none-any.whl (154 kB)
Using cached azure_storage_blob-12.17.0-py3-none-any.whl (388 kB)
Using cached boto3-1.28.21-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.3-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached fastavro-1.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB)
Using cached google_api_core-2.12.0.dev0-py3-none-any.whl (128 kB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached google_cloud_aiplatform-1.29.0-py2.py3-none-any.whl (2.7 MB)
Using cached google_cloud_bigquery-3.11.4-py2.py3-none-any.whl (219 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.20.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.16.1-py2.py3-none-any.whl (176 kB)
Using cached google_cloud_dlp-3.12.2-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.10.1-py2.py3-none-any.whl (101 kB)
Using cached google_cloud_pubsub-2.18.2-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.4-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.0-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.3-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.82.3-py3-none-any.whl (415 kB)
Using cached joblib-1.3.1-py3-none-any.whl (301 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
Using cached psycopg2_binary-2.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (619 kB)
Using cached pytest-7.4.0-py3-none-any.whl (323 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.6.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (772 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Using cached botocore-1.31.21-py3-none-any.whl (11.1 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Using cached dnspython-2.4.1-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.10.0-py2.py3-none-any.whl (114 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (613 kB)
Using cached grpcio_status-1.57.0rc1-py3-none-any.whl (5.1 kB)
Using cached msal-1.24.0b1-py2.py3-none-any.whl (90 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached websocket_client-1.6.1-py3-none-any.whl (56 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.50.0.dev0-py3-none-any.whl size=3257342 sha256=6f74de82a95ed3844df9ab764f01666cb0afbddf32b6af5f40633f46ca3cecb1
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, attrs, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Attempting uninstall: protobuf
Found existing installation: protobuf 4.24.0rc3
Uninstalling protobuf-4.24.0rc3:
Successfully uninstalled protobuf-4.24.0rc3
Successfully installed PyJWT-2.8.0 apache-beam-2.50.0.dev0 attrs-23.1.0 azure-core-1.28.0 azure-identity-1.14.0b2 azure-storage-blob-12.17.0 boto3-1.28.21 botocore-1.31.21 certifi-2023.7.22 cffi-1.15.1 charset-normalizer-3.2.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.3 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.1 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.2 execnet-2.0.2 fastavro-1.8.2 fasteners-0.18 freezegun-1.2.2 google-api-core-2.12.0.dev0 google-apitools-0.5.31 google-auth-2.22.0 google-auth-httplib2-0.1.0 google-cloud-aiplatform-1.29.0 google-cloud-bigquery-3.11.4 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.20.0 google-cloud-core-2.3.3 google-cloud-datastore-2.16.1 google-cloud-dlp-3.12.2 google-cloud-language-2.10.1 google-cloud-pubsub-2.18.2 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.4 google-cloud-resource-manager-1.10.3 google-cloud-spanner-3.40.0 google-cloud-storage-2.10.0 google-cloud-videointelligence-2.11.3 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.5.0 googleapis-common-protos-1.60.0 greenlet-3.0.0a1 grpc-google-iam-v1-0.12.6 grpcio-status-1.57.0rc1 hdfs-2.7.0 httplib2-0.22.0 hypothesis-6.82.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.1 mock-5.1.0 msal-1.24.0b1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.4 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.7.0 proto-plus-1.22.3 protobuf-4.23.4 psycopg2-binary-2.9.7 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pymongo-4.4.1 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.0 pytest-timeout-2.1.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3 pyyaml-6.0.1 regex-2023.6.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.6.1 scikit-learn-1.3.0 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.2 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.7.1 urllib3-1.26.16 websocket-client-1.6.1 wrapt-1.15.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230717" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808125351.1691507263.464155/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808125351.1691507263.464155/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808125351.1691507263.464155/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808125351.1691507263.464155/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20230808150743465177-2960'
createTime: '2023-08-08T15:07:44.436049Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-08-08_08_07_43-9635749521555000607'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0808125351'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-08-08T15:07:44.436049Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-08-08_08_07_43-9635749521555000607]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-08-08_08_07_43-9635749521555000607
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-08-08_08_07_43-9635749521555000607?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-08_08_07_43-9635749521555000607 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:07:49.370Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:07:51.793Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:07:52.123Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-08_08_07_43-9635749521555000607 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:08:04.197Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:12:57.231Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T15:13:10.559Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-08-08_08_07_43-9635749521555000607 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T20:10:14.021Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-08-08_08_07_43-9635749521555000607.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T20:10:16.071Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-08-08T20:10:16.122Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 758, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-08-08_08_07_43-9635749521555000607?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5h 4m 21s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/o2z5gyrvccl3i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org