You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/10/08 20:01:40 UTC
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1141
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1141/display/redirect>
Changes:
------------------------------------------
[...truncated 50.72 KB...]
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.52.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.2-py3-none-any.whl (392 kB)
Using cached boto3-1.28.62-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.2-py2.py3-none-any.whl (181 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.34.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.87.3-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.62-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-py3-none-any.whl size=3337803 sha256=b03b66ab8c569c583d10e0a3d067011e23792cb8ef5ac80a77504311cd9e0b44
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.2 backports.zoneinfo-0.2.1 boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.2 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.34.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.11.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-status-1.59.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.87.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.7 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1008125342.1696777675.174148/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1008125342.1696777675.174148/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1008125342.1696777675.174148/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1008125342.1696777675.174148/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231008150755175119-8598'
createTime: '2023-10-08T15:07:56.204500Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-08_08_07_55-1771849071512953916'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1008125342'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-08T15:07:56.204500Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-08_08_07_55-1771849071512953916]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-08_08_07_55-1771849071512953916
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-08_08_07_55-1771849071512953916?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-08_08_07_55-1771849071512953916 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T15:08:07.975Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T15:08:15.222Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T15:08:15.501Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-08_08_07_55-1771849071512953916 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T15:08:37.766Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T15:09:39.347Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T20:00:40.177Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-08_08_07_55-1771849071512953916.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T20:00:40.314Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-08T20:00:40.380Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-08_08_07_55-1771849071512953916 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-08_08_07_55-1771849071512953916?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 11s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/hbo3nveymimgg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1165
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1165/display/redirect>
Changes:
------------------------------------------
[...truncated 167.76 KB...]
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.75-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.4-py2.py3-none-any.whl (183 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.36.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.13.0-py2.py3-none-any.whl (222 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.2-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.50-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached zstandard-0.22.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
Using cached botocore-1.31.75-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.13.0-py2.py3-none-any.whl (121 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.2-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16214561 sha256=4bf73ff11658212d6cf13abf827d28c7d7c9c8040c38678157a4a9b176ddefb9
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.75 botocore-1.31.75 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.2 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.9.0 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.4 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.36.0 google-cloud-bigquery-3.13.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.13.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.2 grpcio-status-1.59.2 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.2 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.50 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.22.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1101125317.1698851520.483395/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1101125317.1698851520.483395/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1101125317.1698851520.483395/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1101125317.1698851520.483395/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231101151200484367-7363'
createTime: '2023-11-01T15:12:01.403698Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-11-01_08_12_00-387263050461050031'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1101125317'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-11-01T15:12:01.403698Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-11-01_08_12_00-387263050461050031]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-11-01_08_12_00-387263050461050031
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-11-01_08_12_00-387263050461050031?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-11-01_08_12_00-387263050461050031 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:12:08.476Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:12:11.225Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:12:11.706Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-11-01_08_12_00-387263050461050031 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:12:45.482Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:14:29.220Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:14:47.624Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T15:16:12.229Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T16:48:21.873Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T16:48:33.951Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-11-01T17:23:59.650Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(e6fdfe4d93725ff0): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(9e3ba783e6aedf54): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-11-01_08_12_00-387263050461050031 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-11-01_08_12_00-387263050461050031?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 59m 20s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/jpljuowbr3666
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1164
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1164/display/redirect>
Changes:
------------------------------------------
[...truncated 167.00 KB...]
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.74-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.13.0-py2.py3-none-any.whl (222 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.2-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.50-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.74-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.2-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16213847 sha256=6222c7e0a657579863aa0132cd8486f850762c26f1e6256881e50f01d3d4dee1
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.74 botocore-1.31.74 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.9.0 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.13.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.2 grpcio-status-1.59.2 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.2 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.50 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1031125313.1698765132.383356/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1031125313.1698765132.383356/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1031125313.1698765132.383356/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1031125313.1698765132.383356/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231031151212384376-4947'
createTime: '2023-10-31T15:12:13.384483Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-31_08_12_12-8396828785936655355'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1031125313'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-31T15:12:13.384483Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-31_08_12_12-8396828785936655355]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-31_08_12_12-8396828785936655355
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-31_08_12_12-8396828785936655355?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-31_08_12_12-8396828785936655355 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T15:12:25.045Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T15:12:27.487Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T15:12:27.766Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-31_08_12_12-8396828785936655355 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T15:12:56.420Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T15:14:00.463Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T18:44:43.823Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T18:44:45.013Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-31_08_12_12-8396828785936655355 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T20:00:58.470Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-31_08_12_12-8396828785936655355.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T20:00:58.584Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-31T20:00:58.637Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-31_08_12_12-8396828785936655355?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 10s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/ku7uphmzk6o3o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1163
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1163/display/redirect?page=changes>
Changes:
[noreply] minor prism fixes (#29183)
------------------------------------------
[...truncated 166.64 KB...]
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.73-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.2-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.50-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.73-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16209742 sha256=5249482acc503d0ccadcb3cd5611a6f6c0c74a3dfb04bbe4ce540ab1cc9f7e86
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.73 botocore-1.31.73 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.9.0 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.2 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.50 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1030125308.1698678728.160722/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1030125308.1698678728.160722/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1030125308.1698678728.160722/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1030125308.1698678728.160722/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231030151208161686-9575'
createTime: '2023-10-30T15:12:09.186592Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-30_08_12_08-18204949348691583999'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1030125308'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-30T15:12:09.186592Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-30_08_12_08-18204949348691583999]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-30_08_12_08-18204949348691583999
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-30_08_12_08-18204949348691583999?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-30_08_12_08-18204949348691583999 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T15:12:21.323Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T15:12:38.068Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T15:12:38.420Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-30_08_12_08-18204949348691583999 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T15:12:56.927Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T15:14:28.433Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-30_08_12_08-18204949348691583999 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T20:00:53.553Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-30_08_12_08-18204949348691583999.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T20:00:53.652Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-30T20:00:53.691Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-30_08_12_08-18204949348691583999?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 15s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/4tlbtwoaslg2s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1162
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1162/display/redirect>
Changes:
------------------------------------------
[...truncated 166.69 KB...]
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.73-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.73-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16209576 sha256=4fc7397b5b452df3be84f827eeb58d24b342b7152196a38c51e5aec96c9a5a89
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.73 botocore-1.31.73 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.9.0 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1029125309.1698592314.669997/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1029125309.1698592314.669997/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1029125309.1698592314.669997/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1029125309.1698592314.669997/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231029151154670971-2936'
createTime: '2023-10-29T15:11:55.514823Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-29_08_11_55-12795817276001075689'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1029125309'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-29T15:11:55.514823Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-29_08_11_55-12795817276001075689]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-29_08_11_55-12795817276001075689
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-29_08_11_55-12795817276001075689?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-29_08_11_55-12795817276001075689 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T15:12:01.290Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T15:12:09.689Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T15:12:09.943Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-29_08_11_55-12795817276001075689 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T15:12:34.320Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T15:13:39.238Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-29_08_11_55-12795817276001075689 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T20:00:46.967Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-29_08_11_55-12795817276001075689.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T20:00:47.083Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-29T20:00:47.118Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-29_08_11_55-12795817276001075689?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 7s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/niwsykfwhizzi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1161
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1161/display/redirect>
Changes:
------------------------------------------
[...truncated 166.73 KB...]
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.73-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.73-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16209670 sha256=ab5db7e471f786e319bf84ea63e1569d84827260674d5d50a4cc9e79abc3ae20
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.73 botocore-1.31.73 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.9.0 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1028125309.1698505911.404933/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1028125309.1698505911.404933/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1028125309.1698505911.404933/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1028125309.1698505911.404933/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231028151151405900-3302'
createTime: '2023-10-28T15:11:52.369969Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-28_08_11_51-14838288973383482785'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1028125309'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-28T15:11:52.369969Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-28_08_11_51-14838288973383482785]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-28_08_11_51-14838288973383482785
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-28_08_11_51-14838288973383482785?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-28_08_11_51-14838288973383482785 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T15:11:57.638Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T15:12:00.297Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T15:12:00.559Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-28_08_11_51-14838288973383482785 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T15:12:10.526Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T15:13:35.022Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-28_08_11_51-14838288973383482785 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T20:01:00.125Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-28_08_11_51-14838288973383482785.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T20:01:00.255Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-28T20:01:00.298Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-28_08_11_51-14838288973383482785?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 24s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/6eyanueqp4dgq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1160
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1160/display/redirect>
Changes:
------------------------------------------
[...truncated 166.69 KB...]
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0-py3-none-any.whl (164 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.72-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.72-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16209548 sha256=d35f4fee0a39d1dd981a7f2b74cc4a9021d9b810ffaff1a7a2e76383acbf03d5
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.72 botocore-1.31.72 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1027125312.1698419550.887037/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1027125312.1698419550.887037/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1027125312.1698419550.887037/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1027125312.1698419550.887037/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231027151230888049-9107'
createTime: '2023-10-27T15:12:31.838286Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-27_08_12_31-15826178874467118387'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1027125312'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-27T15:12:31.838286Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-27_08_12_31-15826178874467118387]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-27_08_12_31-15826178874467118387
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-27_08_12_31-15826178874467118387?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-27_08_12_31-15826178874467118387 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T15:12:37.549Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T15:12:41.049Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T15:12:41.334Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-27_08_12_31-15826178874467118387 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T15:13:02.188Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T15:14:12.823Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-27_08_12_31-15826178874467118387 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T20:01:08.785Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-27_08_12_31-15826178874467118387.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T20:01:08.919Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-27T20:01:08.960Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-27_08_12_31-15826178874467118387?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 13s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/tpm6w5awlwac4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1159
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1159/display/redirect>
Changes:
------------------------------------------
[...truncated 166.92 KB...]
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.71-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Downloading orjson-3.9.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.5/138.5 kB 6.1 MB/s eta 0:00:00
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.71-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16209240 sha256=7916dcaf918a84b54a6e120978b5da3c33139c12dd8ca177bef7072a03ae9312
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.71 botocore-1.31.71 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.10 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1026125309.1698333140.702410/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1026125309.1698333140.702410/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1026125309.1698333140.702410/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1026125309.1698333140.702410/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231026151220703408-1086'
createTime: '2023-10-26T15:12:21.677756Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-26_08_12_21-14643067112410974340'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1026125309'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-26T15:12:21.677756Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-26_08_12_21-14643067112410974340]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-26_08_12_21-14643067112410974340
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-26_08_12_21-14643067112410974340?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-26_08_12_21-14643067112410974340 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T15:12:32.974Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T15:12:45.008Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T15:12:45.283Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-26_08_12_21-14643067112410974340 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T15:13:11.058Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T15:14:16.303Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-26_08_12_21-14643067112410974340 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T20:00:43.621Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-26_08_12_21-14643067112410974340.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T20:00:43.707Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-26T20:00:43.753Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-26_08_12_21-14643067112410974340?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 5s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/7d43jpbcucu2a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1158
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1158/display/redirect>
Changes:
------------------------------------------
[...truncated 167.48 KB...]
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Downloading greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.7 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.70-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.3-py3-none-any.whl (325 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.70-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Downloading greenlet-3.0.1-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (618 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 618.6/618.6 kB 10.8 MB/s eta 0:00:00
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16207130 sha256=fa50bcfbc43eb9cd8eabd0286e8c20a92178392fc2e61aec2e98ef45a9311d71
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.70 botocore-1.31.70 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.5 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.1 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.3 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1025125312.1698246730.629521/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1025125312.1698246730.629521/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1025125312.1698246730.629521/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1025125312.1698246730.629521/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231025151210630628-7982'
createTime: '2023-10-25T15:12:11.565661Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-25_08_12_11-15862768479812034062'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1025125312'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-25T15:12:11.565661Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-25_08_12_11-15862768479812034062]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-25_08_12_11-15862768479812034062
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-25_08_12_11-15862768479812034062?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-25_08_12_11-15862768479812034062 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T15:12:17.457Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T15:12:20.308Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T15:12:20.626Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-25_08_12_11-15862768479812034062 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T15:12:42.979Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T15:13:47.827Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
WARNING:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T16:27:01.808Z: JOB_MESSAGE_WARNING: Autoscaling: Unable to reach resize target in zone us-central1-b. QUOTA_EXCEEDED: Instance 'load-tests-python-dataflo-10250812-hu8f-harness-pc9x' creation failed: Quota 'DISKS_TOTAL_GB' exceeded. Limit: 500000.0 in region us-central1.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T16:44:28.814Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T16:44:30.656Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-25T18:22:59.598Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(6b2512f8b45b221e): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(e114374760053658): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-25_08_12_11-15862768479812034062 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-25_08_12_11-15862768479812034062?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 54m 22s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/ixux5c3dfwjwo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1157
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1157/display/redirect>
Changes:
------------------------------------------
[...truncated 166.61 KB...]
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.69-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.69-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16207604 sha256=07203456bf1b1847322ea27e8a54548e9b8dff6573767e061d8d0d2e2a91dbd4
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.69 botocore-1.31.69 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231023" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1024125349.1698160381.519878/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1024125349.1698160381.519878/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1024125349.1698160381.519878/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1024125349.1698160381.519878/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231024151301520930-1503'
createTime: '2023-10-24T15:13:02.634808Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-24_08_13_01-7279541586646437243'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1024125349'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-24T15:13:02.634808Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-24_08_13_01-7279541586646437243]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-24_08_13_01-7279541586646437243
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-24_08_13_01-7279541586646437243?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-24_08_13_01-7279541586646437243 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T15:13:09.110Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T15:13:18.227Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T15:13:18.468Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-24_08_13_01-7279541586646437243 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T15:13:32.699Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T15:14:40.320Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-24_08_13_01-7279541586646437243 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T20:01:14.642Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-24_08_13_01-7279541586646437243.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T20:01:14.740Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-24T20:01:14.780Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-24_08_13_01-7279541586646437243?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 15s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/w3feg4zs2g47m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1156
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1156/display/redirect>
Changes:
------------------------------------------
[...truncated 166.68 KB...]
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.68-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pyhamcrest-2.1.0-py3-none-any.whl (54 kB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.1/11.1 MB 82.5 MB/s eta 0:00:00
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.68-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16207509 sha256=f9d815c5f715893809605f1d13d9430f91e32dfef6910cb98cd00e4160f8634e
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.68 botocore-1.31.68 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.1.0 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.2 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.2 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1023125340.1698073935.580293/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1023125340.1698073935.580293/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1023125340.1698073935.580293/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1023125340.1698073935.580293/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231023151215581270-7181'
createTime: '2023-10-23T15:12:16.527857Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-23_08_12_16-8225558607977094487'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1023125340'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-23T15:12:16.527857Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-23_08_12_16-8225558607977094487]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-23_08_12_16-8225558607977094487
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-23_08_12_16-8225558607977094487?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-23_08_12_16-8225558607977094487 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T15:12:27.733Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T15:12:30.887Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T15:12:31.143Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-23_08_12_16-8225558607977094487 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T15:13:00.382Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T15:13:59.655Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T20:00:43.228Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-23_08_12_16-8225558607977094487.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T20:00:43.330Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-23T20:00:43.368Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-23_08_12_16-8225558607977094487 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-23_08_12_16-8225558607977094487?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 21s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/74tt3goai2dtu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1155
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1155/display/redirect>
Changes:
------------------------------------------
[...truncated 167.31 KB...]
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.68-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.68-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16207473 sha256=05525319bbd59af35c961be994a4b692db3bf42f659f6180b1cc0fc25009fc9d
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.68 botocore-1.31.68 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1022125346.1697987526.788733/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1022125346.1697987526.788733/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1022125346.1697987526.788733/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1022125346.1697987526.788733/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231022151206789690-2063'
createTime: '2023-10-22T15:12:07.943511Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-22_08_12_07-14559941820312025029'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1022125346'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-22T15:12:07.943511Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-22_08_12_07-14559941820312025029]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-22_08_12_07-14559941820312025029
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-22_08_12_07-14559941820312025029?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-22_08_12_07-14559941820312025029 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T15:12:14.343Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T15:12:23.736Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T15:12:23.997Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-22_08_12_07-14559941820312025029 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T15:12:47.026Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T15:13:47.672Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T17:48:54.444Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(577af801ded41827): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(12285a94b9079fa3): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T20:00:42.694Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-22_08_12_07-14559941820312025029.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T20:00:42.816Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-22T20:00:42.856Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-22_08_12_07-14559941820312025029 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-22_08_12_07-14559941820312025029?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 54m 27s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/qja7mae7zv2ww
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1154
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1154/display/redirect>
Changes:
------------------------------------------
[...truncated 166.53 KB...]
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0)
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting importlib-resources>=1.4.0 (from jsonschema<5.0.0,>=4.0.0)
Using cached importlib_resources-6.1.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema<5.0.0,>=4.0.0)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting pkgutil-resolve-name>=1.3.10 (from jsonschema<5.0.0,>=4.0.0)
Using cached pkgutil_resolve_name-1.3.10-py3-none-any.whl (4.7 kB)
Collecting referencing>=0.28.4 (from jsonschema<5.0.0,>=4.0.0)
Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema<5.0.0,>=4.0.0)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=1.4.0->jsonschema<5.0.0,>=4.0.0)
Using cached zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.68-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached jsonschema-4.19.1-py3-none-any.whl (83 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.68-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached importlib_resources-6.1.0-py3-none-any.whl (33 kB)
Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached referencing-0.30.2-py3-none-any.whl (25 kB)
Using cached rpds_py-0.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16207526 sha256=aaa7d134c982ee33096bc2cba16866f20d852c3370135fdc91a1556e9cde60de
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, zipp, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, rpds-py, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, pkgutil-resolve-name, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, referencing, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, importlib-resources, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, jsonschema-specifications, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, jsonschema, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, msal, google-cloud-core, boto3, apache-beam, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.68 botocore-1.31.68 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 importlib-resources-6.1.0 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 pkgutil-resolve-name-1.3.10 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 referencing-0.30.2 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rpds-py-0.10.6 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zipp-3.17.0 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1021125345.1697901153.291168/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1021125345.1697901153.291168/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1021125345.1697901153.291168/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1021125345.1697901153.291168/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231021151233292192-6272'
createTime: '2023-10-21T15:12:34.275628Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-21_08_12_33-7783818429490540884'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1021125345'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-21T15:12:34.275628Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-21_08_12_33-7783818429490540884]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-21_08_12_33-7783818429490540884
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-21_08_12_33-7783818429490540884?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-21_08_12_33-7783818429490540884 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T15:12:40.498Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T15:12:42.829Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T15:12:43.107Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-21_08_12_33-7783818429490540884 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T15:12:55.856Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T15:14:07.180Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-21_08_12_33-7783818429490540884 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T20:00:46.504Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-21_08_12_33-7783818429490540884.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T20:00:48.578Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-21T20:00:48.627Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-21_08_12_33-7783818429490540884?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 8s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/s7ojoxfqap6fg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1153
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1153/display/redirect>
Changes:
------------------------------------------
[...truncated 165.16 KB...]
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl.metadata (6.1 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0)
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.5-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.67-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.5-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.23.0rc1-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.67-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16206773 sha256=779adec401d3e51c30b033e0b21369b3dfc0152216d29ca1779d1ee0cf29b07c
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.5 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.67 botocore-1.31.67 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.5 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.23.0rc1 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
WARNING: There was an error checking the latest version of pip.
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1020125341.1697814748.095976/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1020125341.1697814748.095976/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1020125341.1697814748.095976/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1020125341.1697814748.095976/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231020151228096989-4180'
createTime: '2023-10-20T15:12:29.043900Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-20_08_12_28-13182311716805489772'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1020125341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-20T15:12:29.043900Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-20_08_12_28-13182311716805489772]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-20_08_12_28-13182311716805489772
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-20_08_12_28-13182311716805489772?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-20_08_12_28-13182311716805489772 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T15:12:39.270Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T15:12:50.444Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T15:12:51.820Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T15:12:52.107Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-20_08_12_28-13182311716805489772 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T15:14:20.686Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-20_08_12_28-13182311716805489772 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T20:00:47.749Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-20_08_12_28-13182311716805489772.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T20:00:47.875Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-20T20:00:47.917Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-20_08_12_28-13182311716805489772?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 20s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/jmfadzpxesd2o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1152
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1152/display/redirect>
Changes:
------------------------------------------
[...truncated 166.78 KB...]
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.66-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.66-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16206918 sha256=37ca366a10087a778fe28652eb3e916978ec57a17443432dbc262cc3fd0b6570
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.66 botocore-1.31.66 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
WARNING: There was an error checking the latest version of pip.
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1019125341.1697728352.508053/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1019125341.1697728352.508053/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1019125341.1697728352.508053/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1019125341.1697728352.508053/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231019151232509038-3620'
createTime: '2023-10-19T15:12:33.618624Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-19_08_12_32-14734917503801090023'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1019125341'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-19T15:12:33.618624Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-19_08_12_32-14734917503801090023]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-19_08_12_32-14734917503801090023
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-19_08_12_32-14734917503801090023?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-19_08_12_32-14734917503801090023 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:12:40.478Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:12:43.276Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:12:43.591Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-19_08_12_32-14734917503801090023 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:13:14.610Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:14:13.801Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T15:37:22.855Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(2863c131b6695964): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(d18888c2a6af4924): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-19T17:20:36.063Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(eb1bdeff32a44f9d): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(d295b3a44139cfca): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-19_08_12_32-14734917503801090023 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-19_08_12_32-14734917503801090023?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 54m 26s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/6xu5wth7wxmpw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1151
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1151/display/redirect>
Changes:
------------------------------------------
[...truncated 165.18 KB...]
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl.metadata (6.1 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0)
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.19.0b1-py3-none-any.whl (394 kB)
Using cached boto3-1.28.65-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.65-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200556 sha256=1c6956ec2388bf90f8c198ccc4e908669bdb27671209aaada5de258b2ef75e48
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.19.0b1 backports.zoneinfo-0.2.1 boto3-1.28.65 botocore-1.31.65 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.18 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
WARNING: There was an error checking the latest version of pip.
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1018125350.1697641931.597235/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1018125350.1697641931.597235/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1018125350.1697641931.597235/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1018125350.1697641931.597235/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231018151211598209-3139'
createTime: '2023-10-18T15:12:12.614001Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-18_08_12_11-9049160639923779275'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1018125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-18T15:12:12.614001Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-18_08_12_11-9049160639923779275]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-18_08_12_11-9049160639923779275
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-18_08_12_11-9049160639923779275?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-18_08_12_11-9049160639923779275 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T15:12:18.556Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T15:12:26.381Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T15:12:26.527Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-18_08_12_11-9049160639923779275 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T15:12:43.236Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T15:13:51.017Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-18_08_12_11-9049160639923779275 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T20:00:47.557Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-18_08_12_11-9049160639923779275.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T20:00:47.658Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-18T20:00:47.698Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-18_08_12_11-9049160639923779275?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 15s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/if35ft2gkgngg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1150
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1150/display/redirect>
Changes:
------------------------------------------
[...truncated 166.04 KB...]
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0)
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.64-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.1-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.64-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200474 sha256=fbc3c0e55f4453ebc2a100727f9f93783db9d6004addab576180c66fc95e2fdf
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.64 botocore-1.31.64 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.1 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
WARNING: There was an error checking the latest version of pip.
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1017125347.1697555506.758521/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1017125347.1697555506.758521/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1017125347.1697555506.758521/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1017125347.1697555506.758521/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231017151146759477-7611'
createTime: '2023-10-17T15:11:47.831222Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-17_08_11_47-1185265036401412661'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1017125347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-17T15:11:47.831222Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-17_08_11_47-1185265036401412661]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-17_08_11_47-1185265036401412661
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-17_08_11_47-1185265036401412661?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-17_08_11_47-1185265036401412661 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T15:11:53.151Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T15:11:56.074Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T15:11:56.306Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-17_08_11_47-1185265036401412661 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T15:12:21.162Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T15:13:27.448Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T19:31:09.677Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0_sibling_1 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(326a663481ae0934): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(602bcb65af8e9cca): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T20:00:38.350Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-17_08_11_47-1185265036401412661.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T20:00:38.441Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-17T20:00:38.480Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-17_08_11_47-1185265036401412661 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-17_08_11_47-1185265036401412661?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 54m 19s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/4qpt2wpeuiql4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1149
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1149/display/redirect?page=changes>
Changes:
[noreply] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#29002)
------------------------------------------
[...truncated 163.76 KB...]
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting oauth2client>=1.4.12 (from google-apitools<0.5.32,>=0.5.31)
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.18.0)
Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.18.0)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-cloud-storage<3.0.0dev,>=1.32.0 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl.metadata (6.1 kB)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0)
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8)
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0)
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74)
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0)
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0)
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0)
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0)
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0)
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers<4.0.0,>=3.0.3->testcontainers[mysql]<4.0.0,>=3.0.3; extra == "test")
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT<3,>=1.0.0 (from PyJWT[crypto]<3,>=1.0.0->msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0)
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0)
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.63-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.88.0-py3-none-any.whl (421 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.63-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200210 sha256=9896eb2a996f077dcec6b7c91991d17a3e6477f449757fa80c32800012754cb3
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.63 botocore-1.31.63 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.88.0 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
WARNING: There was an error checking the latest version of pip.
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1016125345.1697469113.891932/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1016125345.1697469113.891932/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1016125345.1697469113.891932/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1016125345.1697469113.891932/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231016151153892935-6038'
createTime: '2023-10-16T15:11:55.002338Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-16_08_11_54-922891843405630483'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1016125345'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-16T15:11:55.002338Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-16_08_11_54-922891843405630483]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-16_08_11_54-922891843405630483
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-16_08_11_54-922891843405630483?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-16_08_11_54-922891843405630483 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T15:12:01.595Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T15:12:05.685Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T15:12:06.030Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-16_08_11_54-922891843405630483 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T15:12:15.584Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T15:13:30.572Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-16_08_11_54-922891843405630483 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T18:30:18.993Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-16_08_11_54-922891843405630483.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T18:30:20.922Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-16T18:30:20.974Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-16_08_11_54-922891843405630483?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 3h 24m 19s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/hpixtoijr7em4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1148
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1148/display/redirect>
Changes:
------------------------------------------
[...truncated 184.74 KB...]
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.63-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.87.4-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.63-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200180 sha256=57e751630435c35fb0d616f2a1a425964530d0dcd654802a439580748a06933d
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.63 botocore-1.31.63 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.87.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1015125348.1697382714.256555/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1015125348.1697382714.256555/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1015125348.1697382714.256555/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1015125348.1697382714.256555/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231015151154257504-9310'
createTime: '2023-10-15T15:11:55.344643Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-15_08_11_54-8219740811220299116'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1015125348'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-15T15:11:55.344643Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-15_08_11_54-8219740811220299116]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-15_08_11_54-8219740811220299116
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-15_08_11_54-8219740811220299116?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-15_08_11_54-8219740811220299116 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T15:12:02.108Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T15:12:04.560Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T15:12:04.825Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-15_08_11_54-8219740811220299116 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T15:12:18.879Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T15:13:26.786Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-15_08_11_54-8219740811220299116 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T18:25:15.350Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-15_08_11_54-8219740811220299116.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T18:25:15.438Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-15T18:25:15.488Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-15_08_11_54-8219740811220299116?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 3h 19m 47s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/l43pxndpzqwxm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1147
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1147/display/redirect>
Changes:
------------------------------------------
[...truncated 185.99 KB...]
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.63-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.87.4-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.63-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200097 sha256=3149264ea724efb757b108943cf6b870a99bcc923a786d57282feae06561b202
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.63 botocore-1.31.63 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.87.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1014125337.1697296324.499722/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1014125337.1697296324.499722/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1014125337.1697296324.499722/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1014125337.1697296324.499722/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231014151204500702-8014'
createTime: '2023-10-14T15:12:05.621544Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-14_08_12_04-4668252496200847216'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1014125337'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-14T15:12:05.621544Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-14_08_12_04-4668252496200847216]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-14_08_12_04-4668252496200847216
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-14_08_12_04-4668252496200847216?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-14_08_12_04-4668252496200847216 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T15:12:16.217Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T15:12:18.439Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T15:12:18.869Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-14_08_12_04-4668252496200847216 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T15:12:25.463Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T15:13:42.906Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T18:56:25.711Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(e41a16364e541a8f): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
ERROR:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T18:56:34.128Z: JOB_MESSAGE_ERROR: generic::aborted: SDK harness sdk-0-0 disconnected. This usually means that the process running the pipeline code has crashed. Inspect the Worker Logs and the Diagnostics tab to determine the cause of the crash.
with MessageCode:
(e41a16364e541a8f): SDK disconnect.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_control_service.cc:215
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T20:00:37.642Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-14_08_12_04-4668252496200847216.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T20:00:37.824Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-14T20:00:37.876Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-14_08_12_04-4668252496200847216 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-14_08_12_04-4668252496200847216?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 54m 16s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/mqvtoxqewe2pg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1146
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1146/display/redirect>
Changes:
------------------------------------------
[...truncated 184.78 KB...]
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3.0.0dev,>=1.32.0->google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.24.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b2-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.63-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached grpcio-1.59.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached hypothesis-6.87.4-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Using cached orjson-3.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.63-py3-none-any.whl (11.3 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.12.0-py2.py3-none-any.whl (120 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached shapely-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-cp38-cp38-linux_x86_64.whl size=16200112 sha256=8cd9a6ec199695175216e75d6fdaa2686156c5b2401721b40287b00525f629c8
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, psycopg2-binary, protobuf, portalocker, parameterized, overrides, orjson, objsize, numpy, mock, joblib, jmespath, iniconfig, idna, grpcio, greenlet, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, shapely, scipy, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, pyarrow, proto-plus, isodate, hypothesis, httplib2, googleapis-common-protos, google-resumable-media, cffi, scikit-learn, requests-mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpcio-status, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b2 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.63 botocore-1.31.63 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.12.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.61.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-1.59.0 grpcio-status-1.59.0 hdfs-2.7.3 httplib2-0.22.0 hypothesis-6.87.4 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 numpy-1.24.4 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.9 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 protobuf-4.24.4 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests-mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.2 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1013141614.1697209971.106959/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1013141614.1697209971.106959/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1013141614.1697209971.106959/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1013141614.1697209971.106959/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231013151251107909-3840'
createTime: '2023-10-13T15:12:52.086803Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-13_08_12_51-16402797051709579747'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1013141614'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-13T15:12:52.086803Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-13_08_12_51-16402797051709579747]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-13_08_12_51-16402797051709579747
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-13_08_12_51-16402797051709579747?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-13_08_12_51-16402797051709579747 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T15:12:58.031Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T15:13:07.235Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-13_08_12_51-16402797051709579747 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T15:13:07.477Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T15:13:27.500Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T15:14:29.476Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-13_08_12_51-16402797051709579747 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T20:00:59.935Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-13_08_12_51-16402797051709579747.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T20:01:00.035Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-13T20:01:00.072Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-13_08_12_51-16402797051709579747?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 24s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/ptckfexi5qrli
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1145
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1145/display/redirect>
Changes:
------------------------------------------
[...truncated 50.74 KB...]
Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached shapely-2.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.52.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.62-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.87.3-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.62-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-py3-none-any.whl size=3340505 sha256=82b2e6f2c170c12831b9a873eaecc6ef6c2eeb8330a88ccad2f0a69fd37766c3
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.11.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-status-1.59.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.87.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.8 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1012125347.1697123276.610025/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1012125347.1697123276.610025/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1012125347.1697123276.610025/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1012125347.1697123276.610025/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231012150756611075-6275'
createTime: '2023-10-12T15:07:58.025038Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-12_08_07_56-18365561118498084789'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1012125347'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-12T15:07:58.025038Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-12_08_07_56-18365561118498084789]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-12_08_07_56-18365561118498084789
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-12_08_07_56-18365561118498084789?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-12_08_07_56-18365561118498084789 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T15:08:05.415Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T15:08:08.076Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-12_08_07_56-18365561118498084789 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T15:08:08.426Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T15:08:22.871Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T15:09:41.122Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-12_08_07_56-18365561118498084789 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T20:00:45.381Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-12_08_07_56-18365561118498084789.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T20:00:45.482Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-12T20:00:45.530Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-12_08_07_56-18365561118498084789?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 27s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/eoaqptpirx7hg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1144
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1144/display/redirect>
Changes:
------------------------------------------
[...truncated 50.70 KB...]
Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached shapely-2.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.52.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.3-py3-none-any.whl (392 kB)
Using cached boto3-1.28.62-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.35.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.87.3-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.62-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-py3-none-any.whl size=3338954 sha256=b98c0a18928cebb49b66f0123e02b512d2ef90ff5ebf23c229961a64a75fe772
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.3 backports.zoneinfo-0.2.1 boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.35.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.11.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-status-1.59.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.87.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.8 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-2.0.1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20231009" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1011145557.1697036883.974741/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1011145557.1697036883.974741/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1011145557.1697036883.974741/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1011145557.1697036883.974741/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231011150803975870-5149'
createTime: '2023-10-11T15:08:04.943324Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-11_08_08_04-3913927649183029939'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1011145557'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-11T15:08:04.943324Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-11_08_08_04-3913927649183029939]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-11_08_08_04-3913927649183029939
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-11_08_08_04-3913927649183029939?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-11_08_08_04-3913927649183029939 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T15:08:14.964Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T15:08:26.936Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T15:08:27.195Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-11_08_08_04-3913927649183029939 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T15:08:45.322Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T15:09:58.194Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-11_08_08_04-3913927649183029939 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T20:00:55.721Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-11_08_08_04-3913927649183029939.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T20:00:55.823Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-11T20:00:55.863Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-11_08_08_04-3913927649183029939?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 21s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/jpxc4tyeipksw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1143
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1143/display/redirect>
Changes:
------------------------------------------
[...truncated 50.76 KB...]
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.52.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.2-py3-none-any.whl (392 kB)
Using cached boto3-1.28.62-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.34.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.87.3-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.62-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-py3-none-any.whl size=3338713 sha256=eebe5a2d369df677ad23da09bb520c2700236560747e179bd45bb960c37aff5c
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.2 backports.zoneinfo-0.2.1 boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.3 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.34.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.11.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-status-1.59.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.87.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.7 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1010125348.1696950476.566005/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1010125348.1696950476.566005/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1010125348.1696950476.566005/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1010125348.1696950476.566005/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231010150756566942-2485'
createTime: '2023-10-10T15:07:57.506149Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-10_08_07_56-5693713630432065285'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1010125348'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-10T15:07:57.506149Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-10_08_07_56-5693713630432065285]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-10_08_07_56-5693713630432065285
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-10_08_07_56-5693713630432065285?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-10_08_07_56-5693713630432065285 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T15:08:03.013Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T15:08:05.478Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T15:08:05.776Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-10_08_07_56-5693713630432065285 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T15:08:35.106Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T15:09:37.608Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T20:00:46.033Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-10_08_07_56-5693713630432065285.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T20:00:46.156Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-10T20:00:46.208Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-10_08_07_56-5693713630432065285 is in state JOB_STATE_CANCELLING
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-10_08_07_56-5693713630432065285?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 55m 19s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/3a6maa2ywtjuq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #1142
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/1142/display/redirect?page=changes>
Changes:
[noreply] Bump org.nosphere.apache.rat from 0.8.0 to 0.8.1 (#28675)
------------------------------------------
[...truncated 50.70 KB...]
Collecting shapely<2.0.0 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam==2.52.0.dev0)
Using cached Shapely-1.8.5.post1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for google-resumable-media<3.0dev,>=0.6.0 from https://files.pythonhosted.org/packages/c7/4f/b8e5e22406e5aeafa46df8799939d5eeee52f18eeed339675167fac6603a/google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam==2.52.0.dev0)
Using cached grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for grpcio-status>=1.33.2 from https://files.pythonhosted.org/packages/55/ce/e6d0382610240439ced22fe2183bcc387946bf80e5e0f17f5b5250978ff3/grpcio_status-1.59.0-py3-none-any.whl.metadata
Using cached grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)
Collecting overrides<7.0.0,>=6.0.1 (from google-cloud-pubsublite<2,>=1.2.0->apache-beam==2.52.0.dev0)
Using cached overrides-6.5.0-py3-none-any.whl (17 kB)
Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam==2.52.0.dev0)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam==2.52.0.dev0)
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.23.0,>=0.8->apache-beam==2.52.0.dev0)
Obtaining dependency information for pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 from https://files.pythonhosted.org/packages/39/92/8486ede85fcc088f1b3dba4ce92dd29d126fd96b0008ea213167940a2475/pyparsing-3.1.1-py3-none-any.whl.metadata
Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting attrs>=19.2.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting sortedcontainers<3.0.0,>=2.1.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0 (from hypothesis<=7.0.0,>5.0.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for exceptiongroup>=1.0.0 from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
Using cached exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Obtaining dependency information for tzlocal>=1.2 from https://files.pythonhosted.org/packages/1c/af/343114b3ed9500b46108b56569b31a3108d3669d4fd063d9640e2c36cd57/tzlocal-5.1-py3-none-any.whl.metadata
Using cached tzlocal-5.1-py3-none-any.whl.metadata (16 kB)
Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached pyjsparser-2.7.1-py3-none-any.whl
Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for dnspython<3.0.0,>=1.16.0 from https://files.pythonhosted.org/packages/f6/b4/0a9bee52c50f226a3cbfb54263d02bb421c7f2adc136520729c2c689c1e5/dnspython-2.4.2-py3-none-any.whl.metadata
Using cached dnspython-2.4.2-py3-none-any.whl.metadata (4.9 kB)
Collecting iniconfig (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0)
Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (1.3.0)
Requirement already satisfied: tomli>=1.0.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.8/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.52.0.dev0) (2.0.1)
Collecting execnet>=1.1 (from pytest-xdist<4,>=2.5.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for execnet>=1.1 from https://files.pythonhosted.org/packages/e8/9c/a079946da30fac4924d92dbc617e5367d454954494cf1e71567bcc4e00ee/execnet-2.0.2-py3-none-any.whl.metadata
Using cached execnet-2.0.2-py3-none-any.whl.metadata (2.9 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/1e/c8/fd52271326c052f95f47ef718b018aa2bc3fd097d9bac44d7d48894c6130/charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
Using cached urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting scipy>=1.5.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Using cached scipy-1.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Collecting threadpoolctl>=2.0.0 (from scikit-learn>=0.20.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for threadpoolctl>=2.0.0 from https://files.pythonhosted.org/packages/81/12/fd4dea011af9d69e1cad05c75f3f7202cdcbeac9b712eea58ca779a72865/threadpoolctl-3.2.0-py3-none-any.whl.metadata
Using cached threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy<2.0,>=1.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for greenlet!=0.4.17 from https://files.pythonhosted.org/packages/75/5a/4ecc17749c54882dd3422befcd35fec5128ae2a72138844899322af0b55a/greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Collecting docker>=4.0.0 (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for docker>=4.0.0 from https://files.pythonhosted.org/packages/db/be/3032490fa33b36ddc8c4b1da3252c6f974e7133f1a50de00c6b85cca203a/docker-6.1.3-py3-none-any.whl.metadata
Using cached docker-6.1.3-py3-none-any.whl.metadata (3.5 kB)
Collecting wrapt (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for wrapt from https://files.pythonhosted.org/packages/a6/48/c52023d1446741268a35c60d7f1bfc23060e8683ec581121049f4183507a/wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting deprecation (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql (from testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for pymysql from https://files.pythonhosted.org/packages/e5/30/20467e39523d0cfc2b6227902d3687a16364307260c75e6a1cb4422b0c62/PyMySQL-1.1.0-py3-none-any.whl.metadata
Using cached PyMySQL-1.1.0-py3-none-any.whl.metadata (4.4 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl.metadata
Using cached urllib3-1.26.17-py2.py3-none-any.whl.metadata (48 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.2->apache-beam==2.52.0.dev0)
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0 (from docker>=4.0.0->testcontainers[mysql]<4.0.0,>=3.0.3->apache-beam==2.52.0.dev0)
Obtaining dependency information for websocket-client>=0.32.0 from https://files.pythonhosted.org/packages/c4/3c/1892ce394828c43d4f65248ebdee3854114266b75d1f5915cb211155ad7b/websocket_client-1.6.4-py3-none-any.whl.metadata
Using cached websocket_client-1.6.4-py3-none-any.whl.metadata (7.7 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<4,>=2.0.0->apache-beam==2.52.0.dev0)
Using cached google_crc32c-1.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Collecting PyJWT[crypto]<3,>=1.0.0 (from msal<2.0.0,>=1.20.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for PyJWT[crypto]<3,>=1.0.0 from https://files.pythonhosted.org/packages/2b/4f/e04a8067c7c96c364cef7ef73906504e2f40d690811c021e1a1901473a19/PyJWT-2.8.0-py3-none-any.whl.metadata
Using cached PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting portalocker<3,>=1.0 (from msal-extensions<2.0.0,>=0.3.0->azure-identity<2,>=1.12.0->apache-beam==2.52.0.dev0)
Obtaining dependency information for portalocker<3,>=1.0 from https://files.pythonhosted.org/packages/17/9e/87671efcca80ba6203811540ed1f9c0462c1609d2281d7b7f53cef05da3d/portalocker-2.8.2-py3-none-any.whl.metadata
Using cached portalocker-2.8.2-py3-none-any.whl.metadata (8.5 kB)
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.12->google-apitools<0.5.32,>=0.5.31->apache-beam==2.52.0.dev0)
Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting backports.zoneinfo (from tzlocal>=1.2->js2py<1,>=0.74->apache-beam==2.52.0.dev0)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached azure_core-1.29.4-py3-none-any.whl (192 kB)
Using cached azure_identity-1.15.0b1-py3-none-any.whl (162 kB)
Using cached azure_storage_blob-12.18.2-py3-none-any.whl (392 kB)
Using cached boto3-1.28.62-py3-none-any.whl (135 kB)
Using cached cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
Using cached fastavro-1.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Using cached google_api_core-2.12.0-py3-none-any.whl (121 kB)
Using cached google_auth-2.23.2-py2.py3-none-any.whl (181 kB)
Using cached google_auth_httplib2-0.1.1-py2.py3-none-any.whl (9.3 kB)
Using cached google_cloud_aiplatform-1.34.0-py2.py3-none-any.whl (3.1 MB)
Using cached google_cloud_bigquery-3.12.0-py2.py3-none-any.whl (220 kB)
Using cached google_cloud_bigquery_storage-2.22.0-py2.py3-none-any.whl (190 kB)
Using cached google_cloud_bigtable-2.21.0-py2.py3-none-any.whl (293 kB)
Using cached google_cloud_core-2.3.3-py2.py3-none-any.whl (29 kB)
Using cached google_cloud_datastore-2.18.0-py2.py3-none-any.whl (177 kB)
Using cached google_cloud_dlp-3.12.3-py2.py3-none-any.whl (143 kB)
Using cached google_cloud_language-2.11.1-py2.py3-none-any.whl (138 kB)
Using cached google_cloud_pubsub-2.18.4-py2.py3-none-any.whl (265 kB)
Using cached google_cloud_pubsublite-1.8.3-py2.py3-none-any.whl (288 kB)
Using cached google_cloud_recommendations_ai-0.10.5-py2.py3-none-any.whl (173 kB)
Using cached google_cloud_spanner-3.40.1-py2.py3-none-any.whl (332 kB)
Using cached google_cloud_videointelligence-2.11.4-py2.py3-none-any.whl (229 kB)
Using cached google_cloud_vision-3.4.4-py2.py3-none-any.whl (444 kB)
Using cached hypothesis-6.87.3-py3-none-any.whl (420 kB)
Using cached joblib-1.3.2-py3-none-any.whl (302 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Using cached orjson-3.9.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
Using cached proto_plus-1.22.3-py3-none-any.whl (48 kB)
Using cached psycopg2_binary-2.9.9-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Using cached pymongo-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
Using cached pytest-7.4.2-py3-none-any.whl (324 kB)
Using cached pytest_timeout-2.2.0-py3-none-any.whl (13 kB)
Using cached pytest_xdist-3.3.1-py3-none-any.whl (41 kB)
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Using cached PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Using cached regex-2023.10.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (776 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached requests_mock-1.11.0-py2.py3-none-any.whl (28 kB)
Using cached scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.1 MB)
Using cached SQLAlchemy-1.4.49-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Using cached tenacity-8.2.3-py3-none-any.whl (24 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached botocore-1.31.62-py3-none-any.whl (11.2 MB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached cffi-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (444 kB)
Using cached charset_normalizer-3.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (137 kB)
Using cached dnspython-2.4.2-py3-none-any.whl (300 kB)
Using cached docker-6.1.3-py3-none-any.whl (148 kB)
Using cached exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Using cached execnet-2.0.2-py3-none-any.whl (37 kB)
Using cached google_cloud_resource_manager-1.10.4-py2.py3-none-any.whl (320 kB)
Using cached google_cloud_storage-2.11.0-py2.py3-none-any.whl (118 kB)
Using cached google_resumable_media-2.6.0-py2.py3-none-any.whl (80 kB)
Using cached googleapis_common_protos-1.60.0-py2.py3-none-any.whl (227 kB)
Using cached greenlet-3.0.0-cp38-cp38-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (617 kB)
Using cached grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Using cached msal-1.24.1-py2.py3-none-any.whl (95 kB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached s3transfer-0.7.0-py3-none-any.whl (79 kB)
Using cached threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Using cached tzlocal-5.1-py3-none-any.whl (21 kB)
Using cached urllib3-1.26.17-py2.py3-none-any.whl (143 kB)
Using cached PyMySQL-1.1.0-py3-none-any.whl (44 kB)
Using cached wrapt-1.16.0rc1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Using cached portalocker-2.8.2-py3-none-any.whl (17 kB)
Using cached websocket_client-1.6.4-py3-none-any.whl (57 kB)
Using cached PyJWT-2.8.0-py3-none-any.whl (22 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.52.0.dev0-py3-none-any.whl size=3337803 sha256=89e2685ce3a652368bc2a935e84d5ef95e47ed0c3bce202e06b49b14e1e026f5
Stored in directory: /home/jenkins/.cache/pip/wheels/67/34/b8/5adce605a0a3f10491be234729fa4854c6a127bc01194f13fc
Successfully built apache-beam
Installing collected packages: sortedcontainers, pytz, pyjsparser, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, threadpoolctl, tenacity, sqlparse, six, shapely, scipy, regex, pyyaml, pyparsing, pymysql, PyJWT, pyhamcrest, pycparser, pyasn1, pyarrow, psycopg2-binary, proto-plus, portalocker, parameterized, overrides, orjson, objsize, mock, joblib, jmespath, iniconfig, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, exceptiongroup, dnspython, dill, deprecation, cloudpickle, charset-normalizer, certifi, backports.zoneinfo, attrs, tzlocal, sqlalchemy, scikit-learn, rsa, requests, python-dateutil, pytest, pymongo, pydot, pyasn1-modules, isodate, hypothesis, httplib2, grpcio-status, google-resumable-media, cffi, requests_mock, pytest-xdist, pytest-timeout, pandas, oauth2client, js2py, hdfs, grpc-google-iam-v1, google-auth, freezegun, docker, cryptography, botocore, azure-core, testcontainers, s3transfer, google-auth-httplib2, google-apitools, google-api-core, azure-storage-blob, apache-beam, msal, google-cloud-core, boto3, msal-extensions, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-cloud-pubsublite, google-cloud-aiplatform, azure-identity
Successfully installed PyJWT-2.8.0 apache-beam-2.52.0.dev0 attrs-23.1.0 azure-core-1.29.4 azure-identity-1.15.0b1 azure-storage-blob-12.18.2 backports.zoneinfo-0.2.1 boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 cloudpickle-2.2.1 crcmod-1.7 cryptography-41.0.4 deprecation-2.1.0 dill-0.3.1.1 dnspython-2.4.2 docker-6.1.3 docopt-0.6.2 exceptiongroup-1.1.3 execnet-2.0.2 fastavro-1.8.4 fasteners-0.19 freezegun-1.2.2 google-api-core-2.12.0 google-apitools-0.5.31 google-auth-2.23.2 google-auth-httplib2-0.1.1 google-cloud-aiplatform-1.34.0 google-cloud-bigquery-3.12.0 google-cloud-bigquery-storage-2.22.0 google-cloud-bigtable-2.21.0 google-cloud-core-2.3.3 google-cloud-datastore-2.18.0 google-cloud-dlp-3.12.3 google-cloud-language-2.11.1 google-cloud-pubsub-2.18.4 google-cloud-pubsublite-1.8.3 google-cloud-recommendations-ai-0.10.5 google-cloud-resource-manager-1.10.4 google-cloud-spanner-3.40.1 google-cloud-storage-2.11.0 google-cloud-videointelligence-2.11.4 google-cloud-vision-3.4.4 google-crc32c-1.5.0 google-resumable-media-2.6.0 googleapis-common-protos-1.60.0 greenlet-3.0.0 grpc-google-iam-v1-0.12.6 grpcio-status-1.59.0 hdfs-2.7.2 httplib2-0.22.0 hypothesis-6.87.3 idna-3.4 iniconfig-2.0.0 isodate-0.6.1 jmespath-1.0.1 joblib-1.3.2 js2py-0.74 mock-5.1.0 msal-1.24.1 msal-extensions-1.0.0 oauth2client-4.1.3 objsize-0.6.1 orjson-3.9.7 overrides-6.5.0 pandas-1.5.3 parameterized-0.9.0 portalocker-2.8.2 proto-plus-1.22.3 psycopg2-binary-2.9.9 pyarrow-11.0.0 pyasn1-0.5.0 pyasn1-modules-0.3.0 pycparser-2.21 pydot-1.4.2 pyhamcrest-2.0.4 pyjsparser-2.7.1 pymongo-4.5.0 pymysql-1.1.0 pyparsing-3.1.1 pytest-7.4.2 pytest-timeout-2.2.0 pytest-xdist-3.3.1 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 regex-2023.10.3 requests-2.31.0 requests_mock-1.11.0 rsa-4.9 s3transfer-0.7.0 scikit-learn-1.3.1 scipy-1.10.1 shapely-1.8.5.post1 six-1.16.0 sortedcontainers-2.4.0 sqlalchemy-1.4.49 sqlparse-0.4.4 tenacity-8.2.3 testcontainers-3.7.1 threadpoolctl-3.2.0 typing-extensions-4.8.0 tzlocal-5.1 urllib3-1.26.17 websocket-client-1.6.4 wrapt-1.16.0rc1 zstandard-0.21.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.8_sdk:beam-master-20230927" for Docker environment
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1009125350.1696865153.663746/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1009125350.1696865153.663746/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1009125350.1696865153.663746/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-1009125350.1696865153.663746/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20231009152553664709-9124'
createTime: '2023-10-09T15:25:54.800218Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-10-09_08_25_54-4329034137610977054'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-1009125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-10-09T15:25:54.800218Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2023-10-09_08_25_54-4329034137610977054]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2023-10-09_08_25_54-4329034137610977054
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-09_08_25_54-4329034137610977054?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-09_08_25_54-4329034137610977054 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T15:26:01.858Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T15:26:05.552Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T15:26:05.821Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-09_08_25_54-4329034137610977054 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T15:26:19.190Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T15:27:28.631Z: JOB_MESSAGE_BASIC: All ****s have finished the startup processes and began to receive work requests.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2023-10-09_08_25_54-4329034137610977054 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T20:00:48.629Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2023-10-09_08_25_54-4329034137610977054.
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T20:00:48.753Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2023-10-09T20:00:48.793Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
state = self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 763, in wait_until_finish
assert duration or terminated, (
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2023-10-09_08_25_54-4329034137610977054?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 63
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 4h 37m 16s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://ge.apache.org/s/sbchhwk2krats
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org