You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/01/21 17:06:22 UTC
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 516 - Aborted!
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 516 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/516/ to view the results.
Jenkins build is back to normal : beam_LoadTests_Python_Combine_Dataflow_Streaming #717
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/717/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #716
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/716/display/redirect?page=changes>
Changes:
[alexey.inkin] Fix retaining unsaved pipeline options (#22075)
[108862444+oborysevych] removed VladMatyunin from beam collaborators
[anandinguva98] Add stdlib distutils while building the wheels
[noreply] Add information on how to take/close issues in the contribution guide.
[noreply] Skip
[noreply] Persist ghprbPullId parameter in seed job (#22579)
[noreply] Adhoc: Fix logging in Spark runner to avoid unnecessary creation of
[noreply] Improve exception when requested error tag does not exist (#22401)
------------------------------------------
[...truncated 24.61 KB...]
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.48
Using cached botocore-1.27.48-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835214 sha256=caa5cdd83583b3427ab624afa56aad103e483cb14ea876c8c70d314b247cd709
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, regex, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.48 botocore-1.27.48 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.10.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 regex-2022.7.25 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.40 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809170740.1660144168.982992/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809170740.1660144168.982992/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809170740.1660144168.982992/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809170740.1660144168.982992/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 40s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/b7klco6nusr7c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #715
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/715/display/redirect?page=changes>
Changes:
[vlad.matyunin] modifed WithKeys Playground Example
[alexander.zhuravlev] [Playground] Removed banner from Playground header, deleted unused
[shivam] Add example for `Distinct` PTransform
[manitgupta] Fix bug in StructUtils
[noreply] [Playground][Backend][Bug]: Moving the initialization of properties file
[noreply] Bump cloud.google.com/go/bigquery from 1.36.0 to 1.37.0 in /sdks
[noreply] Minor: Clean up an assertion in schemas_test (#22613)
[noreply] Exclude testWithShardedKeyInGlobalWindow on streaming runner v1 (#22593)
[noreply] Pub/Sub Schema Transform Read Provider (#22145)
[noreply] Update BigQuery URI validation to allow more valid URIs through (#22452)
[noreply] Add units tests for SpannerIO (#22428)
[noreply] Bump google.golang.org/api from 0.90.0 to 0.91.0 in /sdks (#22568)
[noreply] Fix for #22631 KafkaIO considers readCommitted() as it would commit back
[noreply] [CdapIO] Add CdapIO dashboard in Grafana (#22641)
------------------------------------------
[...truncated 22.79 KB...]
Collecting parameterized<0.9.0,>=0.7.1
Using cached parameterized-0.8.1-py2.py3-none-any.whl (26 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<8.0,>=7.1.2
Using cached pytest-7.1.2-py3-none-any.whl (297 kB)
Collecting pytest-xdist<3,>=2.5.0
Using cached pytest_xdist-2.5.0-py3-none-any.whl (41 kB)
Collecting pytest-timeout<3,>=2.1.0
Using cached pytest_timeout-2.1.0-py3-none-any.whl (12 kB)
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.40-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.47
Using cached botocore-1.27.47-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835174 sha256=46ed310f2b373e9777b14b8f6e32adfdc3c094beb8d88d00ce5e89ed2fcb274b
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, regex, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.47 botocore-1.27.47 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.10.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 regex-2022.7.25 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.40 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809150152.1660057651.063864/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809150152.1660057651.063864/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809150152.1660057651.063864/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0809150152.1660057651.063864/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220809150731064848-9035'
createTime: '2022-08-09T15:07:31.922926Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-09_08_07_31-15457082362838978403'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0809150152'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-09T15:07:31.922926Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-09_08_07_31-15457082362838978403]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-09_08_07_31-15457082362838978403
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-09_08_07_31-15457082362838978403?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-09_08_07_31-15457082362838978403 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:35.544Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.486Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.519Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.586Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.671Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.698Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.786Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.850Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.897Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.931Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.952Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:36.983Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.013Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.047Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.083Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.105Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.129Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.157Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.194Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.248Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.269Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.365Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.406Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.434Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.460Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.488Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.543Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:37.571Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:39.623Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-09_08_07_31-15457082362838978403 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:07:48.444Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:08:12.789Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T15:08:46.709Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T16:01:27.112Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T16:02:27.926Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T16:50:30.889Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T16:51:34.469Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T17:37:34.666Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T17:39:35.668Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T18:17:39.040Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T18:28:40.797Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T19:00:43.365Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T19:16:44.769Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T19:43:46.573Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-09_08_07_31-15457082362838978403 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.451Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-09_08_07_31-15457082362838978403.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.485Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.539Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.566Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.591Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-09T20:00:50.618Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-09_08_07_31-15457082362838978403?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 34s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/lpcghndroa6gq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #714
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/714/display/redirect?page=changes>
Changes:
[noreply] Add PyDoc buttons to the top and bottom of the Machine Learning page
------------------------------------------
[...truncated 22.88 KB...]
Using cached google_cloud_pubsublite-1.4.2-py2.py3-none-any.whl (265 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.34.4-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-bigquery-storage<2.14,>=2.6.3
Using cached google_cloud_bigquery_storage-2.13.2-py2.py3-none-any.whl (180 kB)
Collecting google-cloud-core<3,>=0.28.1
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.46
Using cached botocore-1.27.46-py3-none-any.whl (9.0 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835170 sha256=cb2430cb4b34de580c92900ca93bce18692a1fa9bf6969d543009099d952ae64
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.46 botocore-1.27.46 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.10.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808150152.1659971260.100595/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808150152.1659971260.100595/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808150152.1659971260.100595/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0808150152.1659971260.100595/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220808150740101537-2396'
createTime: '2022-08-08T15:07:41.151384Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-08_08_07_40-2217083952252872873'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0808150152'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-08T15:07:41.151384Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-08_08_07_40-2217083952252872873]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-08_08_07_40-2217083952252872873
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-08_08_07_40-2217083952252872873?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-08_08_07_40-2217083952252872873 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:58.174Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.208Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.295Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.376Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.449Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.478Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.552Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.630Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.660Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.695Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.752Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.776Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.807Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.833Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.860Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.881Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.906Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.938Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:07:59.976Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.017Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.078Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.316Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.351Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.380Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.427Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.502Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.558Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.599Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:00.652Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-08_08_07_40-2217083952252872873 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:07.996Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:08:38.560Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T15:09:08.792Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T16:01:53.987Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T16:02:54.521Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T16:50:56.622Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T16:51:57.066Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T17:38:00.811Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T18:25:00.629Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T19:12:04.761Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T19:58:08.234Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:41.896Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-08_08_07_40-2217083952252872873.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:41.925Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:41.977Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:41.995Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:42.035Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:00:42.068Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-08_08_07_40-2217083952252872873 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:01:15.455Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:01:15.534Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-08T20:01:15.554Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-08_08_07_40-2217083952252872873 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-08_08_07_40-2217083952252872873?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/p3bzm3fgh4ify
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #713
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/713/display/redirect>
Changes:
------------------------------------------
[...truncated 22.76 KB...]
Using cached google_cloud_pubsub-2.13.4-py2.py3-none-any.whl (234 kB)
Collecting google-cloud-pubsublite<2,>=1.2.0
Using cached google_cloud_pubsublite-1.4.2-py2.py3-none-any.whl (265 kB)
Collecting google-cloud-bigquery<3,>=1.6.0
Using cached google_cloud_bigquery-2.34.4-py2.py3-none-any.whl (206 kB)
Collecting google-cloud-bigquery-storage<2.14,>=2.6.3
Using cached google_cloud_bigquery_storage-2.13.2-py2.py3-none-any.whl (180 kB)
Collecting google-cloud-core<3,>=0.28.1
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.46
Using cached botocore-1.27.46-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835170 sha256=987286cd9c254ff508b501497dd5547774c22f23c48410f4344e1ffdc900b029
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.46 botocore-1.27.46 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.10.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807150143.1659884860.982985/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807150143.1659884860.982985/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807150143.1659884860.982985/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0807150143.1659884860.982985/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220807150740983944-6687'
createTime: '2022-08-07T15:07:42.857191Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-07_08_07_41-3101032337513237698'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0807150143'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-07T15:07:42.857191Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-07_08_07_41-3101032337513237698]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-07_08_07_41-3101032337513237698
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-07_08_07_41-3101032337513237698?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-07_08_07_41-3101032337513237698 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:46.712Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.134Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.164Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.220Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.303Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.332Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.414Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.469Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.512Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.538Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.563Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.595Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.629Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.653Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.686Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.707Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.729Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.816Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.843Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.877Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:50.903Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.010Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.048Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.135Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.168Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.201Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.262Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.291Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:07:51.357Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-07_08_07_41-3101032337513237698 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:08:17.361Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:08:27.736Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T15:08:54.297Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T16:02:43.030Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T16:48:42.439Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T17:36:46.689Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T18:23:46.718Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T19:10:47.858Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T19:55:53.113Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.544Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-07_08_07_41-3101032337513237698.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.575Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.624Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.645Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.669Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:00:38.689Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-07_08_07_41-3101032337513237698 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:01:20.298Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:01:20.331Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-07T20:01:20.349Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-07_08_07_41-3101032337513237698 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-07_08_07_41-3101032337513237698?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 5s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/bhtoxnqjrsr3e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #712
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/712/display/redirect?page=changes>
Changes:
[yathu] Moving misplaced CHANGES from template to 2.41.0
[noreply] [BEAM-14117] Delete vendored bytebuddy gradle build (#22594)
[noreply] Add Import transform to Go FhirIO (#22460)
[noreply] Allow unsafe triggers for python nexmark benchmarks (#22596)
[noreply] pubsublite: Fix max offset for computing backlog (#22585)
[noreply] Add support when writing to locked buckets by handling
[noreply] [BEAM-14118, #21639] Vendor gRPC 1.48.1 (#22607)
[noreply] [21894] Validates inference_args early (#22282)
[noreply] Return type for _ExpandIntoRanges DoFn should be Iterable. (#22548)
------------------------------------------
[...truncated 23.10 KB...]
Using cached google_cloud_bigquery_storage-2.13.2-py2.py3-none-any.whl (180 kB)
Collecting google-cloud-core<3,>=0.28.1
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.46
Using cached botocore-1.27.46-py3-none-any.whl (9.0 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835170 sha256=6beae6525746115c4f3535574716486836802968e1e20d158050790f8eb0a877
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.46 botocore-1.27.46 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.10.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0806150144.1659798463.001959/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0806150144.1659798463.001959/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0806150144.1659798463.001959/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0806150144.1659798463.001959/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220806150743002917-9849'
createTime: '2022-08-06T15:07:44.180122Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-06_08_07_43-13132476202381895583'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0806150144'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-06T15:07:44.180122Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-06_08_07_43-13132476202381895583]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-06_08_07_43-13132476202381895583
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-06_08_07_43-13132476202381895583?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-06_08_07_43-13132476202381895583 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-06_08_07_43-13132476202381895583 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:47.534Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.352Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.379Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.429Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.515Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.542Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.661Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.714Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.754Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.785Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.811Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.843Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.880Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.916Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.940Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.965Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:48.991Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.016Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.041Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.076Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.103Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.176Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.206Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.225Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.255Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.279Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.334Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.362Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:07:49.391Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:08:12.703Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:08:26.775Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T15:08:53.925Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T16:02:40.622Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T16:03:43.232Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T16:49:43.349Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T16:54:46.712Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T17:27:46.762Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T17:41:48.664Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T18:08:51.838Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T18:27:54.022Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T18:52:55.121Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T19:13:57.652Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T19:36:59.434Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T19:59:01.966Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:41.951Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-06_08_07_43-13132476202381895583.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:41.977Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:42.021Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:42.080Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:42.112Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:00:42.134Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-06_08_07_43-13132476202381895583 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:01:22.103Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:01:22.153Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-06T20:01:22.173Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-06_08_07_43-13132476202381895583 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-06_08_07_43-13132476202381895583?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 3s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/muvhmx22toym2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #711
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/711/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #22347: [22188]Set allowed timestamp skew
[noreply] Added experimental annotation to fixes #22564 (#22565)
------------------------------------------
[...truncated 22.76 KB...]
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<8.0,>=7.1.2
Using cached pytest-7.1.2-py3-none-any.whl (297 kB)
Collecting pytest-xdist<3,>=2.5.0
Using cached pytest_xdist-2.5.0-py3-none-any.whl (41 kB)
Collecting pytest-timeout<3,>=2.1.0
Using cached pytest_timeout-2.1.0-py3-none-any.whl (12 kB)
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.1-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.25.0-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting botocore<1.28.0,>=1.27.46
Using cached botocore-1.27.46-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.2.0-py3-none-any.whl (17 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2835007 sha256=88fb1e401f51efa2112ee3b1e316e075193042384aa73788beb784522872b40a
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, overrides, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.25.0 azure-storage-blob-12.13.1 boto3-1.24.46 botocore-1.27.46 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.2.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0805150319.1659712067.272128/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0805150319.1659712067.272128/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0805150319.1659712067.272128/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0805150319.1659712067.272128/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220805150747273159-8210'
createTime: '2022-08-05T15:07:48.273885Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-05_08_07_47-4951013129220779796'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0805150319'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-05T15:07:48.273885Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-05_08_07_47-4951013129220779796]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-05_08_07_47-4951013129220779796
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-05_08_07_47-4951013129220779796?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-05_08_07_47-4951013129220779796 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:52.313Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.582Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.616Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.675Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.738Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.765Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.842Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.899Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.942Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:53.976Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.011Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.042Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.077Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.109Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.142Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.179Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.201Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.261Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.280Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.311Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.333Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.419Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.452Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.498Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.550Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.582Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.631Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.650Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:07:54.682Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-05_08_07_47-4951013129220779796 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:08:08.429Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:08:30.606Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T15:08:59.887Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T16:02:45.140Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T16:03:55.939Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T16:43:49.165Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T16:48:50.516Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T16:53:50.744Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T17:28:55.561Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T17:43:56.268Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T18:11:19.229Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T18:31:00.014Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T18:54:00.712Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T19:38:04.236Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T19:42:04.797Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-05_08_07_47-4951013129220779796 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:06.860Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-05_08_07_47-4951013129220779796.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:06.899Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:06.961Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:06.978Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:07.010Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-05T20:01:07.038Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-05_08_07_47-4951013129220779796?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 15s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/q2bmmnuh5nekm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #710
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/710/display/redirect?page=changes>
Changes:
[noreply] Update run_inference_basic.ipynb
[noreply] Update CHANGE.md after 2.41.0 cut (#22577)
[noreply] Convert to BeamSchema type from ReadfromBQ (#17159)
[noreply] Fix deleteTimer in InMemoryTimerInternals and enable VR tests for
[noreply] Update Dataflow container version (#22580)
------------------------------------------
[...truncated 22.26 KB...]
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<8.0,>=7.1.2
Using cached pytest-7.1.2-py3-none-any.whl (297 kB)
Collecting pytest-xdist<3,>=2.5.0
Using cached pytest_xdist-2.5.0-py3-none-any.whl (41 kB)
Collecting pytest-timeout<3,>=2.1.0
Using cached pytest_timeout-2.1.0-py3-none-any.whl (12 kB)
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Collecting boto3>=1.9
Using cached boto3-1.24.45-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.45
Using cached botocore-1.27.45-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2834859 sha256=aa7f0ae3c85394c513866691ec067f08b7513e25809bcd57157651679abb4b10
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.45 botocore-1.27.45 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220803" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0804150148.1659625659.181337/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0804150148.1659625659.181337/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0804150148.1659625659.181337/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0804150148.1659625659.181337/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220804150739182289-5210'
createTime: '2022-08-04T15:07:40.096258Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-04_08_07_39-11584301817244931703'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0804150148'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-04T15:07:40.096258Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-04_08_07_39-11584301817244931703]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-04_08_07_39-11584301817244931703
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-04_08_07_39-11584301817244931703?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-04_08_07_39-11584301817244931703 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:46.428Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.244Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.322Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.373Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.435Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.454Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.540Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.619Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.657Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.687Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.715Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.740Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.759Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.783Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.808Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.842Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.921Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.954Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:47.988Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.022Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.045Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.192Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.223Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.243Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.265Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.292Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.348Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.382Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:48.414Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-04_08_07_39-11584301817244931703 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:07:58.589Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:08:25.800Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T15:08:53.019Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T16:04:40.959Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T16:05:41.517Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T16:53:45.407Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T17:38:44.844Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T18:27:48.393Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T19:14:48.925Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-04_08_07_39-11584301817244931703 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:51.997Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-04_08_07_39-11584301817244931703.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:52.045Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:52.102Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:52.131Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:52.162Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-04T20:00:52.186Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-04_08_07_39-11584301817244931703?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 9s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/z6saswtekuixq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #709
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/709/display/redirect?page=changes>
Changes:
[Valentyn Tymofieiev] add zstd compression support according to issue 22393
[Valentyn Tymofieiev] Regenerate the container dependencies.
[noreply] Remove normalization in Pytorch Image Segmentation example (#22371)
[chamikaramj] Mention Java RunInference support in the Website
[noreply] Downgrade less informative logs during write to files (#22273)
[noreply] Beam ml notebooks (#22510)
[noreply] Add clearer error message for xlang transforms on teh Go Direct Runner
[noreply] [CdapIO] Add integration tests for CdapIO (Batch) (#22313)
[noreply] Bugfix: Fix broken assertion in PipelineTest (#22485)
------------------------------------------
[...truncated 22.67 KB...]
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<8.0,>=7.1.2
Using cached pytest-7.1.2-py3-none-any.whl (297 kB)
Collecting pytest-xdist<3,>=2.5.0
Using cached pytest_xdist-2.5.0-py3-none-any.whl (41 kB)
Collecting pytest-timeout<3,>=2.1.0
Using cached pytest_timeout-2.1.0-py3-none-any.whl (12 kB)
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.44
Using cached botocore-1.27.44-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2830086 sha256=9eea231eb790b2940778d5814e168fcd9771dc65bf923ec746ce9d66165fd4e4
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, zstandard, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.44 botocore-1.27.44 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1 zstandard-0.18.0
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0803150148.1659539264.706686/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0803150148.1659539264.706686/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0803150148.1659539264.706686/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0803150148.1659539264.706686/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220803150744707686-8775'
createTime: '2022-08-03T15:07:45.692749Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-03_08_07_45-5926293328673164230'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0803150148'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-03T15:07:45.692749Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-03_08_07_45-5926293328673164230]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-03_08_07_45-5926293328673164230
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-03_08_07_45-5926293328673164230?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-03_08_07_45-5926293328673164230 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:07:49.421Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:07:55.234Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.252Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.564Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.635Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.672Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.728Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.814Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.846Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.878Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.911Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.947Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.973Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:00.996Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.029Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.055Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.090Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.122Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.145Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.171Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.202Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.288Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.323Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.353Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.376Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.429Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.483Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.525Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:01.553Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-03_08_07_45-5926293328673164230 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:18.604Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:36.471Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:36.525Z: JOB_MESSAGE_DETAILED: Resized **** pool to 3, though goal was 5. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:08:57.003Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T15:09:03.077Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T16:03:31.737Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T16:05:51.365Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T16:40:54.842Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T16:55:57.637Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T17:23:59.693Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T17:44:01.092Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T18:07:00.989Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T18:31:04.745Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T18:54:07.303Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T19:17:06.075Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T19:40:09.277Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-03_08_07_45-5926293328673164230 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.697Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-03_08_07_45-5926293328673164230.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.733Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.784Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.832Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.848Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-03T20:00:56.870Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-03_08_07_45-5926293328673164230?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 10s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/ktmhdqrmzriju
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #708
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/708/display/redirect?page=changes>
Changes:
[noreply] Exclude grpcio==1.48.0 (#22539)
[noreply] Merge PR #22304 fixing #22331 fixing JDBC IO IT
[noreply] Update pytest to support Python 3.10 (#22055)
[noreply] Update the imprecise link. (#22549)
------------------------------------------
[...truncated 22.74 KB...]
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.43
Using cached botocore-1.27.43-py3-none-any.whl (9.0 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting tomli>=1.0.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting attrs>=19.2.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: py>=1.8.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.11.0)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (2.1.3)
Requirement already satisfied: pluggy<2.0,>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (1.0.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<8.0,>=7.1.2->apache-beam==2.42.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829788 sha256=28e7f80965ad658e9d010952557e3e312197ac209f446140b1411f1c1a5dabbb
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pytz, pyasn1, parameterized, iniconfig, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tomli, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, sqlalchemy, scipy, requests, pyarrow, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Successfully installed apache-beam-2.42.0.dev0 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.43 botocore-1.27.43 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 iniconfig-1.1.1 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-7.1.2 pytest-forked-1.4.0 pytest-timeout-2.1.0 pytest-xdist-2.5.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 tomli-2.0.1 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0802150148.1659452864.218885/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0802150148.1659452864.218885/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0802150148.1659452864.218885/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0802150148.1659452864.218885/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220802150744219905-6785'
createTime: '2022-08-02T15:07:45.154354Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-02_08_07_44-4525323818181222852'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0802150148'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-02T15:07:45.154354Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-02_08_07_44-4525323818181222852]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-02_08_07_44-4525323818181222852
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-02_08_07_44-4525323818181222852?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-02_08_07_44-4525323818181222852 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:50.110Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.047Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.085Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.140Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.213Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.243Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.325Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.391Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.445Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.482Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.555Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.608Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.633Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.660Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.692Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.726Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.761Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.793Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.827Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.860Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:51.893Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.023Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.057Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.086Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.120Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.150Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.202Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:52.232Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:07:54.284Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-02_08_07_44-4525323818181222852 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:08:09.828Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:08:28.126Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T15:08:54.905Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T16:02:23.909Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T16:03:22.523Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T16:40:22.279Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T16:51:25.761Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T17:20:28.177Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T17:30:28.588Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T18:02:32.392Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T18:16:30.892Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T18:45:32.643Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T19:02:35.077Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T19:29:37.913Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T19:48:40.075Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-02_08_07_44-4525323818181222852 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.203Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-02_08_07_44-4525323818181222852.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.247Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.304Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.331Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.357Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:00:46.378Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:01:27.509Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:01:27.556Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-02T20:01:27.593Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-02_08_07_44-4525323818181222852 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-02_08_07_44-4525323818181222852?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/gtwngy2fv3vis
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #707
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/707/display/redirect>
Changes:
------------------------------------------
[...truncated 22.77 KB...]
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting botocore<1.28.0,>=1.27.42
Using cached botocore-1.27.42-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.48.0-py3-none-any.whl (14 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829783 sha256=8965d12864519c616d6a87e03e26894d95a40e2e00715043ecd525fd850d4795
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.42.0.dev0 atomicwrites-1.4.1 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.42 botocore-1.27.42 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.48.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0801150152.1659366462.931498/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0801150152.1659366462.931498/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0801150152.1659366462.931498/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0801150152.1659366462.931498/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220801150742932512-3922'
createTime: '2022-08-01T15:07:43.869293Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-08-01_08_07_43-7907489071298693366'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0801150152'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-08-01T15:07:43.869293Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-08-01_08_07_43-7907489071298693366]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-08-01_08_07_43-7907489071298693366
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-08-01_08_07_43-7907489071298693366?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-01_08_07_43-7907489071298693366 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:48.491Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.422Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.451Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.520Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.593Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.633Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.680Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.769Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.840Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.897Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.934Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.956Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:49.979Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.006Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.027Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.050Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.078Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.112Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.143Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.175Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.200Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.289Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.327Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.360Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.383Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.412Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.471Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.492Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:50.529Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-01_08_07_43-7907489071298693366 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:07:59.652Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:08:26.837Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 4 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:08:26.865Z: JOB_MESSAGE_DETAILED: Resized **** pool to 4, though goal was 5. This could be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:08:37.044Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:08:53.834Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T15:59:41.468Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T16:03:42.465Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T16:37:44.323Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T16:52:47.113Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T17:18:48.434Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T17:36:51.107Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T18:02:53.723Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T18:25:57.411Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T18:49:59.766Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T19:27:00.082Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-08-01_08_07_43-7907489071298693366 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.196Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-08-01_08_07_43-7907489071298693366.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.248Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.331Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.353Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.391Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-08-01T20:01:06.416Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-08-01_08_07_43-7907489071298693366?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/nnr47ergwl6zo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #706
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/706/display/redirect>
Changes:
------------------------------------------
[...truncated 23.17 KB...]
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting botocore<1.28.0,>=1.27.42
Using cached botocore-1.27.42-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.48.0-py3-none-any.whl (14 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting attrs>=17.4.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829783 sha256=faa8fa3e957fbc375afb21bb40d29bdc19032c94042ba9ac65d87c5930ace6ae
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.42.0.dev0 atomicwrites-1.4.1 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.42 botocore-1.27.42 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.48.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.11 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0731150147.1659280050.779087/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0731150147.1659280050.779087/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0731150147.1659280050.779087/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0731150147.1659280050.779087/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220731150730780141-6973'
createTime: '2022-07-31T15:07:31.605437Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-07-31_08_07_31-14573533748014680050'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0731150147'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-07-31T15:07:31.605437Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-07-31_08_07_31-14573533748014680050]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-07-31_08_07_31-14573533748014680050
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-31_08_07_31-14573533748014680050?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-31_08_07_31-14573533748014680050 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:39.660Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:40.758Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:40.787Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:40.861Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:40.931Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:40.961Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.029Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.094Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.132Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.159Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.183Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.239Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.272Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.293Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.318Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.344Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.380Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.411Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.445Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.479Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.532Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.637Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.668Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.697Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.741Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.766Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.823Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.867Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-31_08_07_31-14573533748014680050 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:07:41.901Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:08:02.728Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:08:15.515Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T15:08:40.952Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T16:01:28.889Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T16:04:30.033Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T16:39:33.981Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T16:53:36.188Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T17:21:36.703Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T17:40:39.763Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T18:04:41.543Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T18:26:42.547Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T18:49:45.082Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T19:12:48.051Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T19:34:47.761Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T19:57:50.435Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-31_08_07_31-14573533748014680050 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.559Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-07-31_08_07_31-14573533748014680050.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.585Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.626Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.648Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.671Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:00:41.692Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:01:19.811Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:01:19.844Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-31T20:01:19.870Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-31_08_07_31-14573533748014680050 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-07-31_08_07_31-14573533748014680050?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 11s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/3xmhbs6jraa3c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #705
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/705/display/redirect?page=changes>
Changes:
[noreply] Bump google.golang.org/api from 0.89.0 to 0.90.0 in /sdks (#22518)
[noreply] Change _build import from setuptools to distutils (#22503)
[noreply] Remove stringx package (#22534)
[noreply] Improve concrete error message (#22536)
------------------------------------------
[...truncated 22.90 KB...]
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting botocore<1.28.0,>=1.27.41
Using cached botocore-1.27.41-py3-none-any.whl (9.0 MB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.48.0-py3-none-any.whl (14 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting attrs>=17.4.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (2.1.3)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829783 sha256=9943b3f204f43bc846131a06f75639f8851682be348d65f81879bbb7d4091486
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.42.0.dev0 atomicwrites-1.4.1 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.41 botocore-1.27.41 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.4 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.48.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.10 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0730150144.1659193657.367093/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0730150144.1659193657.367093/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0730150144.1659193657.367093/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0730150144.1659193657.367093/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220730150737368053-1991'
createTime: '2022-07-30T15:07:38.167825Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-07-30_08_07_37-17831505780634411888'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0730150144'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-07-30T15:07:38.167825Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-07-30_08_07_37-17831505780634411888]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-07-30_08_07_37-17831505780634411888
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-30_08_07_37-17831505780634411888?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-30_08_07_37-17831505780634411888 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:41.467Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.117Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.165Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.229Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.288Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.308Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.374Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.418Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.449Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.463Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.485Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.510Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.534Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.628Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.662Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.694Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.727Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.761Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.815Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.848Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.881Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:42.999Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.018Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.049Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.074Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.102Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.157Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.186Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:43.219Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-30_08_07_37-17831505780634411888 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:07:55.002Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:08:20.486Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T15:08:53.946Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T16:03:09.220Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T16:04:10.892Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T16:49:11.156Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T16:51:12.383Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T17:27:14.466Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T17:38:15.750Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T18:10:19.173Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T18:54:19.757Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T18:58:24.606Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T19:28:25.453Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T19:43:26.462Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-30_08_07_37-17831505780634411888 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.361Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-07-30_08_07_37-17831505780634411888.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.397Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.442Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.461Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.487Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-30T20:00:47.511Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-07-30_08_07_37-17831505780634411888?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 6s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/rm5kkebj6bonm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #704
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/704/display/redirect?page=changes>
Changes:
[chamikaramj] Remove unnecessary reference to use_runner_v2 experiment in x-lang
[yixiaoshen] Fix typo in Datastore V1ReadIT test
[noreply] Remove deprecated Session runner (#22505)
[noreply] Add Go test status to the PR template (#22508)
[noreply] Relax the google-api-core dependency. (#22513)
[noreply] Bump google.golang.org/protobuf from 1.28.0 to 1.28.1 in /sdks (#22517)
------------------------------------------
[...truncated 22.71 KB...]
Collecting google-cloud-core<3,>=0.28.1
Using cached google_cloud_core-2.3.2-py2.py3-none-any.whl (29 kB)
Collecting google-cloud-bigtable<2,>=0.31.1
Using cached google_cloud_bigtable-1.7.2-py2.py3-none-any.whl (267 kB)
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.3-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.8.0-py2.py3-none-any.whl (119 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.2-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.40
Using cached botocore-1.27.40-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.48.0-py3-none-any.whl (14 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829699 sha256=497073eefa5e670ac22f2ea7026d3c33732815e8a214ebc1515ec20f87c2684b
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.42.0.dev0 atomicwrites-1.4.1 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.40 botocore-1.27.40 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.48.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0729150148.1659107258.972554/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0729150148.1659107258.972554/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0729150148.1659107258.972554/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0729150148.1659107258.972554/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220729150738973565-9013'
createTime: '2022-07-29T15:07:39.976379Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-07-29_08_07_39-11223459129437769696'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0729150148'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-07-29T15:07:39.976379Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-07-29_08_07_39-11223459129437769696]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-07-29_08_07_39-11223459129437769696
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-29_08_07_39-11223459129437769696?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-29_08_07_39-11223459129437769696 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:48.158Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.399Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.424Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.535Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.643Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.673Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.737Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.798Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.829Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.852Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.875Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.901Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.934Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.967Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:49.999Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.032Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.065Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.089Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.128Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.168Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.190Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.288Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.314Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.334Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.367Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.398Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.455Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.504Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:07:50.547Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-29_08_07_39-11223459129437769696 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:08:24.452Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:08:26.215Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T15:08:50.110Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T16:02:39.059Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T16:05:43.072Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T16:41:41.984Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T16:52:44.758Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T17:15:46.068Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T17:37:49.061Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T18:00:50.740Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T18:23:51.111Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T18:38:52.916Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T19:06:06.103Z: JOB_MESSAGE_WARNING: Autoscaling: Unable to reach resize target in zone us-central1-b. ZONE_RESOURCE_POOL_EXHAUSTED: Instance 'load-tests-python-dataflo-07290807-tswf-harness-llx0' creation failed: The zone 'projects/apache-beam-testing/zones/us-central1-b' does not have enough resources available to fulfill the request. Try a different zone, or try again later.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-29_08_07_39-11223459129437769696 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.385Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-07-29_08_07_39-11223459129437769696.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.475Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.551Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.570Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.592Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-29T20:00:45.613Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-07-29_08_07_39-11223459129437769696?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 4s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/l3wp4rrbrioqk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #703
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/703/display/redirect?page=changes>
Changes:
[bulat.safiullin] [Website] remove beam-summit 2022 container with all related files
[samuelw] Fixes #22438. Ensure that WindmillStateReader completes all batched read
[noreply] Bump google.golang.org/api from 0.88.0 to 0.89.0 in /sdks (#22464)
[noreply] Upgrades pip before installing Beam for Python default expansion service
[noreply] [Go SDK]: Plumb allowed lateness to execution (#22476)
[Valentyn Tymofieiev] Restrict google-api-core
[Valentyn Tymofieiev] Regenerate the container dependencies.
[noreply] Replace distutils with supported modules. (#22456)
[noreply] [22369] Default Metrics for Executable Stages in Samza Runner (#22370)
[Kiley Sok] Moving to 2.42.0-SNAPSHOT on master branch.
[noreply] Remove stripping of step name. Replace removing only suffix step name
[noreply] Add read/write PubSub integration example fhirio pipeline (#22306)
------------------------------------------
[...truncated 22.81 KB...]
Collecting scikit-learn>=0.20.0
Using cached scikit_learn-1.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24.8 MB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.39-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.6.0-py2.py3-none-any.whl (41 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)
Collecting boto3>=1.9
Using cached boto3-1.24.39-py3-none-any.whl (132 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.42.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting botocore<1.28.0,>=1.27.39
Using cached botocore-1.27.39-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.42.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.42.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting attrs>=17.4.0
Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.42.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.42.0.dev0-py3-none-any.whl size=2829736 sha256=81cd88c09b194467458b77b6fbf91832d9b6f7220e54adc2233e70e6b6ed66e0
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.42.0.dev0 atomicwrites-1.4.1 attrs-22.1.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.39 botocore-1.27.39 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.1 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.42.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0728150156.1659020863.504161/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0728150156.1659020863.504161/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0728150156.1659020863.504161/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0728150156.1659020863.504161/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220728150743505125-5052'
createTime: '2022-07-28T15:07:44.462366Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-07-28_08_07_44-12424103367984834527'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0728150156'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-07-28T15:07:44.462366Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-07-28_08_07_44-12424103367984834527]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-07-28_08_07_44-12424103367984834527
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-28_08_07_44-12424103367984834527?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-28_08_07_44-12424103367984834527 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:49.801Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.221Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.255Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.311Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.386Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.417Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.466Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.547Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.577Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.607Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.644Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.670Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.716Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.751Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.785Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.818Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.852Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.901Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.935Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.973Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:51.998Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.098Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.136Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.181Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.214Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.247Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.300Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.333Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:07:52.364Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-28_08_07_44-12424103367984834527 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:08:21.067Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:08:29.458Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T15:08:56.607Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T16:02:18.650Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T16:04:23.550Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T16:34:23.456Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T16:50:21.602Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T17:17:23.982Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T17:37:28.549Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T18:00:28.969Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T18:20:30.904Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T18:40:33.063Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T19:05:35.299Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T19:27:34.898Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T19:50:38.913Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-28_08_07_44-12424103367984834527 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.048Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-07-28_08_07_44-12424103367984834527.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.092Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.146Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.188Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.241Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-28T20:00:51.264Z: JOB_MESSAGE_BASIC: Stopping **** pool...
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1665, in wait_until_finish
'{}'.format(consoleUrl))
AssertionError: Job did not reach to a terminal state after waiting indefinitely. Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-07-28_08_07_44-12424103367984834527?project=<ProjectId>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m 3s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/xclocb25ysc2k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #702
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/702/display/redirect?page=changes>
Changes:
[chamikaramj] Adds KV support for the Java RunInference transform.
[noreply] Replace distutils with supported modules. (#21968)
[noreply] Revert "Replace distutils with supported modules. " (#22453)
[noreply] Enable configuration to avoid successfully written Table Row propagation
[noreply] lint fixes for recent import (#22455)
[noreply] Bump Python Combine LoadTests timeout to 12 hours (#22439)
[noreply] convert windmill min timestamp to beam min timestamp (#21915)
[noreply] [CdapIO] Fixed necessary warnings (#22399)
[noreply] [#22051]: Add read_time support to Google Cloud Datastore connector
[noreply] 21730 fix offset resetting (#22450)
------------------------------------------
[...truncated 23.01 KB...]
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.3-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.2-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<0.8.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.7.0-py2.py3-none-any.whl (148 kB)
Collecting azure-storage-blob>=12.3.2
Using cached azure_storage_blob-12.13.0-py3-none-any.whl (377 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.24.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from azure-core>=1.7.0->apache-beam==2.41.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.7.1-py3-none-any.whl (85 kB)
Collecting botocore<1.28.0,>=1.27.38
Using cached botocore-1.27.38-py3-none-any.whl (9.0 MB)
Collecting jmespath<2.0.0,>=0.7.1
Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.7.0,>=0.6.0
Using cached s3transfer-0.6.0-py3-none-any.whl (79 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting oauth2client>=1.4.12
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.2-py3-none-any.whl (114 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.3.3-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: packaging<22.0dev,>=14.3 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.41.0.dev0) (21.3)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2829681 sha256=b9beb0e063f9c6c5511bc533c99f6913821d1e214845d5f2a770a62bff388b08
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.38 botocore-1.27.38 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-2.8.2 google-apitools-0.5.31 google-auth-2.9.1 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.7.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.11 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0727150150.1658934473.679217/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0727150150.1658934473.679217/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0727150150.1658934473.679217/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0727150150.1658934473.679217/pipeline.pb in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220727150753680239-3596'
createTime: '2022-07-27T15:07:54.960192Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-07-27_08_07_54-9812810296344857627'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0727150150'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-07-27T15:07:54.960192Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-07-27_08_07_54-9812810296344857627]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-07-27_08_07_54-9812810296344857627
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-07-27_08_07_54-9812810296344857627?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-27_08_07_54-9812810296344857627 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:00.442Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-a.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.436Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.501Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.553Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.610Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.638Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.704Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.792Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.833Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.868Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:908>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.898Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:908>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.932Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.957Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:01.991Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.034Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.068Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.091Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.118Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.154Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.178Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.203Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.335Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.357Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.384Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.416Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.448Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.546Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.578Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-a...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:02.623Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-27_08_07_54-9812810296344857627 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:21.634Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:08:39.836Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T15:09:05.490Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T16:01:26.229Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T16:03:30.330Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T16:38:29.927Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T16:50:31.947Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T17:22:34.853Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T17:38:37.773Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T18:02:39.190Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T18:19:40.062Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T18:45:42.510Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T19:04:43.091Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T19:28:45.617Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T19:47:46.879Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.307Z: JOB_MESSAGE_BASIC: Cancel request is committed for workflow job: 2022-07-27_08_07_54-9812810296344857627.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.354Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.419Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.489Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.512Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:00:41.533Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-27_08_07_54-9812810296344857627 is in state JOB_STATE_CANCELLING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:01:16.922Z: JOB_MESSAGE_DETAILED: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:01:16.959Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-07-27T20:01:16.987Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-07-27_08_07_54-9812810296344857627 is in state JOB_STATE_CANCELLED
ERROR:apache_beam.runners.dataflow.dataflow_runner:Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2022-07-27_08_07_54-9812810296344857627?project=<ProjectId>
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/combine_test.py",> line 129, in <module>
CombineTest().run()
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 151, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1676, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: CANCELLED, Error:
None
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 55m
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/z2eqk6ykjbre4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 701 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 701 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/701/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 700 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 700 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/700/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 699 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 699 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/699/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 698 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 698 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/698/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #697
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/697/display/redirect?page=changes>
Changes:
[balazs.nemeth] BEAM-14525 Fix for Protobuf getter/setter method name discovery issue
[balazs.nemeth] BEAM-14525 Added a proto message with the problematic properties to use
[balazs.nemeth] PR CR: updating issue links
[noreply] Add accept-language header for MPL license (#22395)
[noreply] Bump terser from 5.9.0 to 5.14.2 in
[noreply] Fixes #22156: Fix Spark3 runner to compile against Spark 3.2/3.3 and add
[Moritz Mack] Closes #22407: Separate sources for SparkStructuredStreamingRunner for
[Moritz Mack] Add deprecation warning for Spark 2 in SparkStructuredStreamingRunner
------------------------------------------
[...truncated 25.57 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827581 sha256=7512613a669b0acbe8aa618d3a74d66b5ae95e0de8055829cbc0d9912370d78a
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.35 botocore-1.27.35 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0722150159.1658502451.171034/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0722150159.1658502451.171034/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0722150159.1658502451.171034/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0722150159.1658502451.171034/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 6s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/jmnfa3uiiztf2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #696
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/696/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Support combiner lifting.
[noreply] Bump google.golang.org/api from 0.87.0 to 0.88.0 in /sdks (#22350)
[Robert Bradshaw] More clarification.
[noreply] [CdapIO] HasOffset interface was implemented (#22193)
[noreply] added olehborysevych as collaborator (#22391)
------------------------------------------
[...truncated 25.62 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827581 sha256=07be0f7cde7ed8fa4dac587d99ceaa079f439c7b09d538eb1d7212268c1a0577
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.34 botocore-1.27.34 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0721150150.1658416076.184911/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0721150150.1658416076.184911/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0721150150.1658416076.184911/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0721150150.1658416076.184911/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 30s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/p5vc7lovbdie2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #695
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/695/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Require unique names for stages.
[noreply] cleaned up types in standard_coders.ts (#22316)
[noreply] JMH module for sdks:java:core with benchmarks for
[noreply] Bump cloud.google.com/go/pubsub from 1.23.1 to 1.24.0 in /sdks (#22332)
[Luke Cwik] [#22181] Fix java package for SDK java core benchmark
[Luke Cwik] Allow jmhTest to run concurrently with other jmhTest instances
[noreply] [BEAM-13015, #21250] Optimize encoding to a ByteString (#22345)
------------------------------------------
[...truncated 25.66 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827581 sha256=c59a31d2dfabfd21359de4218de2a0405e3d75ab681b2dd1f9258c27e68fcb44
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.33 botocore-1.27.33 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.3 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.9 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0720150159.1658329667.201982/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0720150159.1658329667.201982/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0720150159.1658329667.201982/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0720150159.1658329667.201982/pipeline.pb in 1 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 20s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/hveosn45ecelc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #694
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/694/display/redirect?page=changes>
Changes:
[noreply] [BEAM-14117] Unvendor bytebuddy dependency (#17317)
[noreply] Use npm ci instead of install in CI (#22323)
[noreply] Fix typo in use_single_core_per_container logic. (#22318)
[noreply] [#22319] Regenerate proto2_coder_test_messages_pb2.py manually (#22320)
[noreply] Add links to the new RunInference content to Learning Resources (#22325)
[noreply] Unskip RunInference IT tests (#22324)
------------------------------------------
[...truncated 25.75 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827581 sha256=0e37233aa7adf2ce02ca0878c0f3feef9afa576eb8de4ce4c3e9e85121e12fac
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.32 botocore-1.27.32 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.8 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0719150143.1658243259.530585/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0719150143.1658243259.530585/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0719150143.1658243259.530585/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0719150143.1658243259.530585/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 14s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/okgpgtnfu4q3a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #693
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/693/display/redirect?page=changes>
Changes:
[Alexey Romanenko] [website] Add TPC-DS benchmark documentation
[noreply] Increase streaming server timeout (#22280)
------------------------------------------
[...truncated 25.78 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827423 sha256=418e28c7c2b7535c583b24e5b54a2a6ae12549e93fb67303976ee6283e120985
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.31 botocore-1.27.31 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.8.0 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0718150149.1658156855.083175/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0718150149.1658156855.083175/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0718150149.1658156855.083175/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0718150149.1658156855.083175/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/p22f6h6xpri6a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #692
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/692/display/redirect>
Changes:
------------------------------------------
[...truncated 25.62 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827423 sha256=265c17edec64f9a59310e43ba5e8e30c9eae834ea5d5761f043e88bed2909718
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.31 botocore-1.27.31 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.1 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.4 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0717150141.1658070462.479812/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0717150141.1658070462.479812/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0717150141.1658070462.479812/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0717150141.1658070462.479812/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 12s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/lulbcucu552um
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #691
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/691/display/redirect?page=changes>
Changes:
[noreply] Bump protobufjs from 6.11.2 to 6.11.3 in /sdks/typescript
[vlad.matyunin] enabled multifile flag for multifile examples (PG)
[Robert Bradshaw] Don't try to parse non-flags as retained pipeline options.
[chamikaramj] Enables UnboundedSource wrapped SDF Kafka source by default for x-lang
[noreply] Merge pull request #22140 from [Playground Task] Sharing any code API
[bulat.safiullin] [Website] add playground section, update playground, update get-started
[noreply] RunInference documentation updates. (#22236)
[noreply] Turn pr bot on for remaining common labels (#22257)
[noreply] Reviewing the RunInference ReadMe file for clarity. (#22069)
[noreply] Collect heap profile on OOM on Dataflow (#22225)
[noreply] fixing the missing wrap around ring range read (#21786)
[noreply] Update RunInference documentation (#22250)
[noreply] Rewrote Java multi-language pipeline quickstart (#22263)
[noreply] Merge pull request #22300 from Fixed [Playground] DeployExamples,
------------------------------------------
[...truncated 25.62 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2827423 sha256=4a3bc2ba8a323a0b236326a4d4dbbe4efb4ec47d3c1fd39b402857f4a52953ae
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.31 botocore-1.27.31 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.1 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.3 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0716150146.1657984055.367642/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0716150146.1657984055.367642/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0716150146.1657984055.367642/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0716150146.1657984055.367642/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 11s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/jpt3nqi72sjnk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #690
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/690/display/redirect?page=changes>
Changes:
[vitaly.terentyev] [BEAM-14101] Add Spark Receiver IO package and ReceiverBuilder
[egalpin] Moves timestamp skew override to correct place
[egalpin] Adds TestStream to verify window preservation of ElasticsearchIO#write
[egalpin] Removes unnecessary line
[Heejong Lee] [BEAM-22229] Override external SDK container URLs for Dataflow by
[egalpin] Adds validation that ES#Write outputs are in expected windows
[egalpin] Updates window verification test to assert the exact docs in the window
[egalpin] Uses guava Iterables over shaded avro version
[danthev] Fix query retry in Java FirestoreIO.
[noreply] Pg auth test (#22277)
[noreply] [BEAM-14073] [CdapIO] CDAP IO for batch plugins: Read, Write. Unit tests
[Heejong Lee] update
[noreply] [Fix #22151] Add fhirio.Deidentify transform (#22152)
[noreply] Remove locks around ExecutionStateSampler (#22190)
------------------------------------------
[...truncated 25.66 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2823479 sha256=08c6daa9483333a7d5b3bf497f3017e3da089ec6d8dafd49b2e5d5dda7ada5bd
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.30 botocore-1.27.30 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.1 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.3 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0715150151.1657897664.752398/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0715150151.1657897664.752398/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0715150151.1657897664.752398/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0715150151.1657897664.752398/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 19s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/jscjgmzxgkwao
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #689
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/689/display/redirect?page=changes>
Changes:
[Heejong Lee] [BEAM-14506] Adding testcases and examples for xlang Python RunInference
[Heejong Lee] update
[Heejong Lee] update
[noreply] Move Go Primitives Integration Tests to Generic Registration (#22247)
[noreply] Move native Go examples to generic registration (#22245)
[noreply] Move youngoli to the reviewer exclusion list (#22195)
[noreply] Bump google.golang.org/api from 0.86.0 to 0.87.0 in /sdks (#22253)
[noreply] Bump cloud.google.com/go/bigquery from 1.34.1 to 1.35.0 in /sdks
[noreply] Bump google.golang.org/grpc from 1.47.0 to 1.48.0 in /sdks (#22252)
[noreply] Merge pull request #15786: Add gap-filling transform for timeseries
[chamikaramj] Adds an experiment that allows opting into using Kafka SDF-wrapper
[noreply] Defocus iframe on blur or mouseout (#22153) (#22154)
[noreply] Fix pydoc rendering for annotated classes (#22121)
[noreply] Fix typo in comment (#22266)
[noreply] Split words on new lines or spaces (#22270)
[noreply] Replace \r\n, not just \n
------------------------------------------
[...truncated 25.58 KB...]
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
Using cached googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc,grpcgcp] to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-spanner<2,>=1.13.0->apache-beam==2.41.0.dev0) (63.2.0)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2823479 sha256=34431ce3cbb4f8f46a9067aceaad0ec310d83671fabf55ce9b63efbb39e9d2c5
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.29 botocore-1.27.29 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.1 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.3 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0714150201.1657811256.983414/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0714150201.1657811256.983414/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0714150201.1657811256.983414/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0714150201.1657811256.983414/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/5xvfdqyo33grw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #688
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/688/display/redirect?page=changes>
Changes:
[naireenhussain] add new pubsub urn
[Pablo Estrada] Several requests to show experiments in Dataflow UI
[byronellis] Add org.pentaho to calcite relocated packages to fix vendoring
[noreply] Adding VladMatyunin as collaborator (#22239)
[noreply] Mark session runner as deprecated (#22242)
[noreply] Update google-cloud-core dependency to <3 (#22237)
[noreply] Move WC integration test to generic registration (#22248)
[noreply] Move Xlang Go examples to generic registration (#22249)
------------------------------------------
[...truncated 76.79 KB...]
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of mock to determine which version is compatible with other requirements. This could take a while.
Using cached httplib2-0.20.1-py3-none-any.whl (96 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of hdfs to determine which version is compatible with other requirements. This could take a while.
Collecting hdfs<3.0.0,>=2.1.0
Using cached hdfs-2.6.0-py3-none-any.whl (33 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of joblib to determine which version is compatible with other requirements. This could take a while.
Using cached hdfs-2.5.8.tar.gz (41 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of grpcio-gcp to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of grpcio to determine which version is compatible with other requirements. This could take a while.
Collecting grpcio<2,>=1.33.1
Using cached grpcio-1.47.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5 MB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of httplib2 to determine which version is compatible with other requirements. This could take a while.
Using cached grpcio-1.46.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.4 MB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
INFO: pip is looking at multiple versions of google-cloud-vision to determine which version is compatible with other requirements. This could take a while.
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.1-py2.py3-none-any.whl (435 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Using cached google_cloud_vision-0.42.0-py2.py3-none-any.whl (435 kB)
Using cached google_cloud_vision-0.41.0-py2.py3-none-any.whl (431 kB)
Using cached google_cloud_vision-0.40.0-py2.py3-none-any.whl (431 kB)
Using cached google_cloud_vision-0.39.0-py2.py3-none-any.whl (418 kB)
Using cached google_cloud_vision-0.38.1-py2.py3-none-any.whl (413 kB)
Using cached google_cloud_vision-0.38.0-py2.py3-none-any.whl (413 kB)
INFO: pip is looking at multiple versions of google-cloud-videointelligence to determine which version is compatible with other requirements. This could take a while.
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.2-py2.py3-none-any.whl (183 kB)
INFO: pip is looking at multiple versions of google-cloud-spanner to determine which version is compatible with other requirements. This could take a while.
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.2-py2.py3-none-any.whl (255 kB)
INFO: pip is looking at multiple versions of google-cloud-recommendations-ai to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of google-cloud-pubsublite to determine which version is compatible with other requirements. This could take a while.
Collecting google-cloud-pubsublite<2,>=1.2.0
Using cached google_cloud_pubsublite-1.4.1-py2.py3-none-any.whl (265 kB)
INFO: pip is looking at multiple versions of google-cloud-pubsub to determine which version is compatible with other requirements. This could take a while.
Collecting google-cloud-pubsub<3,>=2.1.0
Using cached google_cloud_pubsub-2.13.1-py2.py3-none-any.whl (234 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.1)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2822936 sha256=499eee30f418c94be25225c442ae692d2fc48846f582fa7c97d24aaf24252833
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.28 botocore-1.27.28 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.32.0 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-2.3.1 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0713150141.1657724979.935813/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0713150141.1657724979.935813/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0713150141.1657724979.935813/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0713150141.1657724979.935813/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 13s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/g6koowxtosg5u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 687 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 687 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/687/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #686
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/686/display/redirect?page=changes>
Changes:
[noreply] Parallelizable DataFrame/Series mean (#22174)
------------------------------------------
[...truncated 25.38 KB...]
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.41.0.dev0) (63.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.3-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2821181 sha256=3c32d852ce968f9576d8e3a48f38651608813c1fb122a35f6f9d0d8139aa613a
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.26 botocore-1.27.26 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.6 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-1.7.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.3 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657552060.736206/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657552060.736206/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657552060.736206/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657552060.736206/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 18s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/tp33a7atyctrq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #685
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/685/display/redirect?page=changes>
Changes:
[noreply] Add typescript documentation to the programing guide. (#22137)
[noreply] [Website] Update minimum required Go version for sdk development
------------------------------------------
[...truncated 25.44 KB...]
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.6-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.41.0.dev0) (63.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.3-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2821134 sha256=3130dc4430ffe595737d6b87fbbe3be87a24efcb3fd1ef1a059ba4e55d189dac
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.26 botocore-1.27.26 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.6 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-1.7.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.3 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657465663.315354/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657465663.315354/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657465663.315354/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657465663.315354/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 17s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/rnb2rqvmdh2we
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #684
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/684/display/redirect?page=changes>
Changes:
[bulat.safiullin] [Website] change case studies link from staging to relative path
[bulat.safiullin] [Website] add I/O Connectors link to dropdown list, updating link to
[noreply] Merge pull request #22096 from [Playground] Infrastructure for sharing
[noreply] Support dependencies and remote registration in the typescript SDK.
[noreply] [BEAM-13015, #22050] Make SDK harness msec counters faster using ordered
[yathu] Fix build error due to dep confliction of google-cloud-bigquery-storage
[yathu] Fix atomicwrites old version purge on pypi
[noreply] Fix default type inference of CombinePerKey. (#16351)
------------------------------------------
[...truncated 25.38 KB...]
Collecting google-api-core<2.0.0dev,>=1.21.0
Using cached google_api_core-1.31.6-py2.py3-none-any.whl (93 kB)
Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from google-auth<3,>=1.18.0->apache-beam==2.41.0.dev0) (63.1.0)
Collecting grpcio-status>=1.16.0
Using cached grpcio_status-1.47.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from httplib2<0.21.0,>=0.8->apache-beam==2.41.0.dev0) (3.0.9)
Collecting pbr>=0.11
Using cached pbr-5.9.0-py2.py3-none-any.whl (112 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.1-py2.py3-none-any.whl
Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (1.11.0)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting wrapt
Using cached wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting docker>=4.0.0
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.56.3-py2.py3-none-any.whl (211 kB)
WARNING: google-api-core 2.8.2 does not provide the extra 'grpcgcp'
Collecting google-api-core[grpc,grpcgcp]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5
Using cached google_api_core-2.8.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.8.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.3-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.2-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.7.0-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.1-py3-none-any.whl (114 kB)
Using cached google_api_core-2.6.0-py2.py3-none-any.whl (114 kB)
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-2.3.2-py2.py3-none-any.whl (109 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.41.0.dev0) (3.8.0)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.41.0.dev0-py3-none-any.whl size=2821108 sha256=065c6d8af44561a654880bec930d4ac3bb765188b6eafc82e9611b28600f261e
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, docopt, crcmod, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, threadpoolctl, tenacity, rsa, pyyaml, python-dateutil, pymysql, pymongo, pyhamcrest, pydot, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, joblib, jmespath, isodate, idna, httplib2, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, certifi, cachetools, attrs, atomicwrites, sqlalchemy, scipy, requests, pyarrow, pluggy, pandas, overrides, oauth2client, mock, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, deprecation, cffi, botocore, scikit-learn, s3transfer, requests-oauthlib, requests_mock, pytest, hdfs, grpc-google-iam-v1, google-auth-httplib2, google-apitools, google-api-core, docker, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.41.0.dev0 atomicwrites-1.4.1 attrs-21.4.0 azure-core-1.24.2 azure-storage-blob-12.13.0 boto3-1.24.26 botocore-1.27.26 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cloudpickle-2.1.0 crcmod-1.7 cryptography-37.0.4 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.5.2 fasteners-0.17.3 freezegun-1.2.1 google-api-core-1.31.6 google-apitools-0.5.31 google-auth-1.35.0 google-auth-httplib2-0.1.0 google-cloud-bigquery-2.34.4 google-cloud-bigquery-storage-2.13.2 google-cloud-bigtable-1.7.2 google-cloud-core-1.7.2 google-cloud-datastore-1.15.5 google-cloud-dlp-3.7.1 google-cloud-language-1.3.2 google-cloud-pubsub-2.13.1 google-cloud-pubsublite-1.4.2 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.3 google-cloud-videointelligence-1.16.3 google-cloud-vision-1.0.2 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.3 greenlet-1.1.2 grpc-google-iam-v1-0.12.4 grpcio-gcp-0.2.2 grpcio-status-1.47.0 hdfs-2.7.0 httplib2-0.20.4 idna-3.3 isodate-0.6.1 jmespath-1.0.1 joblib-1.1.0 mock-2.0.0 more-itertools-8.13.0 msrest-0.7.1 numpy-1.21.6 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.7.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.8.1 pbr-5.9.0 pluggy-0.13.1 proto-plus-1.20.6 psycopg2-binary-2.9.3 pyarrow-7.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2022.1 pyyaml-6.0 requests-2.28.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.6.0 scikit-learn-1.0.2 scipy-1.7.3 sqlalchemy-1.4.39 tenacity-5.1.5 testcontainers-3.6.0 threadpoolctl-3.1.0 typing-extensions-4.3.0 typing-utils-0.1.0 urllib3-1.26.10 wcwidth-0.2.5 websocket-client-1.3.3 wrapt-1.14.1
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz"> to staging location.
INFO:apache_beam.runners.dataflow.dataflow_runner:Pipeline has additional dependencies to be installed in SDK **** container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.41.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220617" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657379246.691299/dataflow_python_sdk.tar...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657379246.691299/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657379246.691299/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0706185343.1657379246.691299/pipeline.pb in 0 seconds.
usage: combine_test.py [-h] [--runner RUNNER] [--streaming]
[--resource_hint RESOURCE_HINTS]
[--beam_services BEAM_SERVICES]
[--type_check_strictness {ALL_REQUIRED,DEFAULT_TO_ANY}]
[--type_check_additional TYPE_CHECK_ADDITIONAL]
[--no_pipeline_type_check] [--runtime_type_check]
[--performance_runtime_type_check]
[--allow_non_deterministic_key_coders]
[--allow_unsafe_triggers]
[--no_direct_runner_use_stacked_bundle]
[--direct_runner_bundle_repeat DIRECT_RUNNER_BUNDLE_REPEAT]
[--direct_num_****s DIRECT_NUM_WORKERS]
[--direct_running_mode {in_memory,multi_threading,multi_processing}]
[--direct_embed_docker_python]
[--dataflow_endpoint DATAFLOW_ENDPOINT]
[--project PROJECT] [--job_name JOB_NAME]
[--staging_location STAGING_LOCATION]
[--temp_location TEMP_LOCATION] [--region REGION]
[--service_account_email SERVICE_ACCOUNT_EMAIL]
[--no_auth] [--template_location TEMPLATE_LOCATION]
[--label LABELS] [--update]
[--transform_name_mapping TRANSFORM_NAME_MAPPING]
[--enable_streaming_engine]
[--dataflow_kms_key DATAFLOW_KMS_KEY]
[--create_from_snapshot CREATE_FROM_SNAPSHOT]
[--flexrs_goal {COST_OPTIMIZED,SPEED_OPTIMIZED}]
[--dataflow_service_option DATAFLOW_SERVICE_OPTIONS]
[--enable_hot_key_logging] [--enable_artifact_caching]
[--impersonate_service_account IMPERSONATE_SERVICE_ACCOUNT]
[--hdfs_host HDFS_HOST] [--hdfs_port HDFS_PORT]
[--hdfs_user HDFS_USER] [--hdfs_full_urls]
[--num_****s NUM_WORKERS]
[--max_num_****s MAX_NUM_WORKERS]
[--autoscaling_algorithm {NONE,THROUGHPUT_BASED}]
[--****_machine_type MACHINE_TYPE]
[--disk_size_gb DISK_SIZE_GB]
[--****_disk_type DISK_TYPE]
[--****_region WORKER_REGION]
[--****_zone WORKER_ZONE] [--zone ZONE]
[--network NETWORK] [--subnetwork SUBNETWORK]
[--****_harness_container_image WORKER_HARNESS_CONTAINER_IMAGE]
[--sdk_container_image SDK_CONTAINER_IMAGE]
[--sdk_harness_container_image_overrides SDK_HARNESS_CONTAINER_IMAGE_OVERRIDES]
[--default_sdk_harness_log_level DEFAULT_SDK_HARNESS_LOG_LEVEL]
[--sdk_harness_log_level_overrides SDK_HARNESS_LOG_LEVEL_OVERRIDES]
[--use_public_ips] [--no_use_public_ips]
[--min_cpu_platform MIN_CPU_PLATFORM]
[--dataflow_****_jar DATAFLOW_WORKER_JAR]
[--dataflow_job_file DATAFLOW_JOB_FILE]
[--experiment EXPERIMENTS]
[--number_of_****_harness_threads NUMBER_OF_WORKER_HARNESS_THREADS]
[--profile_cpu] [--profile_memory]
[--profile_location PROFILE_LOCATION]
[--profile_sample_rate PROFILE_SAMPLE_RATE]
[--requirements_file REQUIREMENTS_FILE]
[--requirements_cache REQUIREMENTS_CACHE]
[--requirements_cache_only_sources]
[--setup_file SETUP_FILE] [--beam_plugin BEAM_PLUGINS]
[--pickle_library {cloudpickle,default,dill}]
[--save_main_session] [--sdk_location SDK_LOCATION]
[--extra_package EXTRA_PACKAGES]
[--prebuild_sdk_container_engine PREBUILD_SDK_CONTAINER_ENGINE]
[--prebuild_sdk_container_base_image PREBUILD_SDK_CONTAINER_BASE_IMAGE]
[--cloud_build_machine_type CLOUD_BUILD_MACHINE_TYPE]
[--docker_registry_push_url DOCKER_REGISTRY_PUSH_URL]
[--job_endpoint JOB_ENDPOINT]
[--artifact_endpoint ARTIFACT_ENDPOINT]
[--job_server_timeout JOB_SERVER_TIMEOUT]
[--environment_type ENVIRONMENT_TYPE]
[--environment_config ENVIRONMENT_CONFIG]
[--environment_option ENVIRONMENT_OPTIONS]
[--sdk_****_parallelism SDK_WORKER_PARALLELISM]
[--environment_cache_millis ENVIRONMENT_CACHE_MILLIS]
[--output_executable_path OUTPUT_EXECUTABLE_PATH]
[--artifacts_dir ARTIFACTS_DIR] [--job_port JOB_PORT]
[--artifact_port ARTIFACT_PORT]
[--expansion_port EXPANSION_PORT]
[--job_server_java_launcher JOB_SERVER_JAVA_LAUNCHER]
[--job_server_jvm_properties JOB_SERVER_JVM_PROPERTIES]
[--flink_master FLINK_MASTER]
[--flink_version {1.12,1.13,1.14,1.15}]
[--flink_job_server_jar FLINK_JOB_SERVER_JAR]
[--flink_submit_uber_jar]
[--spark_master_url SPARK_MASTER_URL]
[--spark_job_server_jar SPARK_JOB_SERVER_JAR]
[--spark_submit_uber_jar]
[--spark_rest_url SPARK_REST_URL]
[--spark_version {2,3}]
[--on_success_matcher ON_SUCCESS_MATCHER]
[--dry_run DRY_RUN]
[--wait_until_finish_duration WAIT_UNTIL_FINISH_DURATION]
[--pubsub_root_url PUBSUBROOTURL]
[--s3_access_key_id S3_ACCESS_KEY_ID]
[--s3_secret_access_key S3_SECRET_ACCESS_KEY]
[--s3_session_token S3_SESSION_TOKEN]
[--s3_endpoint_url S3_ENDPOINT_URL]
[--s3_region_name S3_REGION_NAME]
[--s3_api_version S3_API_VERSION]
[--s3_verify S3_VERIFY] [--s3_disable_ssl]
[--publish_to_big_query PUBLISH_TO_BIG_QUERY]
[--metrics_dataset METRICS_DATASET]
[--metrics_table METRICS_TABLE]
[--influx_measurement INFLUX_MEASUREMENT]
[--influx_db_name INFLUX_DB_NAME]
[--influx_hostname INFLUX_HOSTNAME]
[--input_options INPUT_OPTIONS]
[--timeout_ms TIMEOUT_MS] [--top_count=20 TOP_COUNT=20]
combine_test.py: error: argument --top_count=20: expected one argument
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 3s
15 actionable tasks: 9 executed, 4 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/ayh2r23yrltdw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 683 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 683 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/683/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 682 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 682 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/682/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 681 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 681 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/681/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 680 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 680 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/680/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 679 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 679 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/679/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 678 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 678 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/678/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 677 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 677 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/677/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 676 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 676 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/676/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 675 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 675 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/675/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 674 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 674 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/674/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 673 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 673 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/673/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 672 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 672 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/672/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 671 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 671 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/671/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 670 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 670 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/670/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 669 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 669 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/669/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 668 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 668 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/668/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 667 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 667 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/667/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 666 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 666 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/666/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 665 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 665 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/665/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 664 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 664 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/664/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 663 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 663 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/663/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 662 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 662 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/662/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 661 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 661 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/661/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 660 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 660 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/660/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 659 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 659 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/659/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 658 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 658 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/658/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 657 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 657 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/657/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 656 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 656 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/656/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 655 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 655 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/655/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 654 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 654 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/654/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 653 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 653 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/653/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 652 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 652 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/652/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 651 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 651 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/651/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 650 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 650 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/650/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 649 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 649 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/649/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 648 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 648 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/648/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 647 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 647 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/647/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 646 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 646 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/646/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 645 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 645 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/645/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 644 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 644 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/644/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 643 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 643 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/643/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 642 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 642 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/642/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 641 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 641 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/641/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 640 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 640 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/640/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 639 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 639 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/639/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 638 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 638 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/638/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 637 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 637 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/637/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 636 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 636 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/636/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 635 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 635 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/635/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 634 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 634 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/634/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 633 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 633 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/633/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 632 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 632 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/632/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 631 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 631 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/631/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 630 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 630 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/630/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 629 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 629 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/629/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 628 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 628 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/628/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 627 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 627 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/627/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 626 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 626 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/626/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 625 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 625 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/625/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 624 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 624 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/624/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 623 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 623 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/623/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 622 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 622 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/622/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 621 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 621 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/621/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 620 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 620 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/620/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 619 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 619 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/619/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 618 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 618 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/618/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 617 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 617 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/617/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 616 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 616 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/616/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 615 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 615 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/615/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 614 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 614 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/614/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 613 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 613 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/613/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 612 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 612 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/612/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 611 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 611 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/611/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 610 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 610 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/610/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 609 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 609 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/609/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 608 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 608 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/608/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 607 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 607 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/607/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 606 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 606 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/606/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 605 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 605 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/605/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 604 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 604 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/604/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 603 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 603 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/603/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 602 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 602 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/602/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 601 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 601 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/601/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 600 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 600 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/600/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 599 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 599 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/599/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 598 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 598 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/598/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 597 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 597 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/597/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 596 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 596 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/596/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 595 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 595 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/595/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 594 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 594 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/594/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 593 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 593 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/593/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 592 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 592 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/592/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 591 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 591 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/591/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 590 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 590 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/590/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 589 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 589 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/589/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 588 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 588 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/588/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 587 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 587 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/587/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 586 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 586 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/586/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 585 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 585 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/585/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 584 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 584 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/584/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 583 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 583 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/583/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 582 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 582 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/582/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 581 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 581 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/581/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 580 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 580 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/580/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 579 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 579 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/579/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 578 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 578 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/578/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 577 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 577 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/577/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 576 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 576 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/576/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 575 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 575 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/575/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 574 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 574 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/574/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 573 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 573 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/573/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 572 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 572 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/572/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 571 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 571 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/571/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 570 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 570 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/570/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 569 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 569 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/569/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 568 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 568 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/568/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 567 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 567 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/567/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 566 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 566 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/566/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 565 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 565 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/565/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 564 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 564 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/564/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 563 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 563 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/563/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 562 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 562 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/562/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 561 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 561 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/561/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 560 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 560 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/560/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 559 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 559 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/559/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 558 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 558 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/558/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 557 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 557 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/557/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 556 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 556 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/556/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 555 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 555 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/555/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 554 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 554 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/554/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 553 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 553 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/553/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 552 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 552 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/552/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 551 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 551 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/551/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 550 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 550 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/550/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 549 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 549 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/549/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/547/display/redirect>
Changes:
------------------------------------------
[...truncated 22.35 KB...]
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting pandas<2.0.0
Using cached pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting boto3>=1.9
Using cached boto3-1.21.4-py3-none-any.whl (132 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting botocore<1.25.0,>=1.24.4
Using cached botocore-1.24.4-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.3)
Collecting libcst>=0.3.10
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.44.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2675864 sha256=6e4ff095baaf8288ad2868e5f49221b36e31d2822aa7e2f66c73c945d7130a89
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.21.4 botocore-1.24.4 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.34.0 google-cloud-bigquery-storage-2.12.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.4.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.44.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.7 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.3 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.38.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220208
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220208" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0222150558.1645542448.046933/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0222150558.1645542448.046933/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0222150558.1645542448.046933/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0222150558.1645542448.046933/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--top_count=20', 'shuffle_mode=appliance']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--top_count=20', 'shuffle_mode=appliance']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220222150728047878-6991'
createTime: '2022-02-22T15:07:28.936001Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-22_07_07_28-4826287301479849318'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0222150558'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-22T15:07:28.936001Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-22_07_07_28-4826287301479849318]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-22_07_07_28-4826287301479849318
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-22_07_07_28-4826287301479849318?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-22_07_07_28-4826287301479849318 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:36.945Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.654Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.686Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.766Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.835Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.873Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:38.942Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.007Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.050Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.086Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:898>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.119Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:898>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.151Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.186Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.221Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.264Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.297Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.322Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.356Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.378Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.444Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.474Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.599Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.627Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.647Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.677Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.710Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.772Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.803Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:39.843Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-22_07_07_28-4826287301479849318 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:07:56.119Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:08:24.546Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:08:50.014Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:08:50.046Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T15:51:30.766Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-22T16:28:34.300Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-6' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy121.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-6 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 546 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 546 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/546/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 545 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 545 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/545/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 544 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 544 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/544/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 543 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 543 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/543/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 542 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 542 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/542/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/540/display/redirect?page=changes>
Changes:
[akustov] fix name project id from secreton scio deploy action
[alexander.zhuravlev] [BEAM-13775] Fixed bug with run button
[ihr] [BEAM-13836] Fix the answers placeholders locations in the Python katas
[noreply] Merge pull request #16703 from [BEAM-13804][Playground][Bugfix] Add
[noreply] Merge pull request #16611 from [BEAM-13712][Playground] Add graph for
[noreply] Merge pull request #16757 from [BEAM-13655] [Playground] Persist the
------------------------------------------
[...truncated 13.30 KB...]
Collecting google-cloud-spanner<2,>=1.13.0
Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB)
Collecting grpcio-gcp<1,>=0.2.2
Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB)
Collecting google-cloud-dlp<4,>=3.0.0
Using cached google_cloud_dlp-3.6.0-py2.py3-none-any.whl (111 kB)
Collecting google-cloud-language<2,>=1.3.0
Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB)
Collecting google-cloud-videointelligence<2,>=1.8.0
Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB)
Collecting google-cloud-vision<2,>=0.38.0
Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB)
Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0
Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers[mysql]<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Collecting cryptography>=36.0.0
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.38.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.1-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.54
Using cached botocore-1.23.54-py3-none-any.whl (8.5 MB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.3-py3-none-any.whl (18 kB)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.2.1-py2.py3-none-any.whl (75 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.38.0.dev0) (21.3)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.5.0-py2.py3-none-any.whl (111 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.38.0.dev0) (60.9.0)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.1-py2.py3-none-any.whl (113 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (2.1.3)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (1.11.0)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting pymysql
Using cached PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2659546 sha256=dbaa32da7a42edd9693bb6832fe4753bb57bc551b4ff3be3b384f9e3af875c01
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: wcwidth, pytz, pyasn1, parameterized, mypy-extensions, docopt, crcmod, certifi, wrapt, websocket-client, urllib3, typing-utils, typing-extensions, tenacity, rsa, pyyaml, python-dateutil, pyparsing, pymysql, pymongo, pyhamcrest, pycparser, pyasn1-modules, psycopg2-binary, proto-plus, pbr, orjson, oauthlib, numpy, more-itertools, jmespath, isodate, idna, greenlet, googleapis-common-protos, google-crc32c, fasteners, fastavro, execnet, dill, cloudpickle, charset-normalizer, cachetools, attrs, atomicwrites, typing-inspect, sqlalchemy, requests, pydot, pyarrow, pluggy, pandas, overrides, mock, httplib2, grpcio-status, grpcio-gcp, google-resumable-media, google-auth, freezegun, cffi, botocore, s3transfer, requests-oauthlib, requests_mock, pytest, oauth2client, libcst, hdfs, grpc-google-iam-v1, google-api-core, docker, deprecation, cryptography, azure-core, testcontainers, pytest-timeout, pytest-forked, msrest, google-cloud-core, google-apitools, boto3, apache-beam, pytest-xdist, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, azure-storage-blob, google-cloud-pubsublite
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.54 botocore-1.23.54 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.2.0 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-oauthlib-1.3.1 requests_mock-1.9.3 rsa-4.8 s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0214125350.1644853176.970921/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220214153936971889-2388'
createTime: '2022-02-14T15:39:37.787025Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-02-14_07_39_37-4298799526588690428'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0214125350'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-02-14T15:39:37.787025Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-02-14_07_39_37-4298799526588690428]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-02-14_07_39_37-4298799526588690428
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-14_07_39_37-4298799526588690428?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-14_07_39_37-4298799526588690428 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:43.773Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.030Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.057Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.127Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.198Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.221Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.288Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.357Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.395Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.421Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:898>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.443Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:898>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.470Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.503Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.530Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.557Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.602Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.634Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.660Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.681Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.718Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.749Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.871Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.904Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.940Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:46.969Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:47.019Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:47.040Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-c...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:39:47.072Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-02-14_07_39_37-4298799526588690428 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:40:19.668Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:40:27.598Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:40:51.914Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T15:40:51.949Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T16:18:41.360Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T16:24:34.443Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T16:52:35.137Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-02-14T17:19:38.222Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-13' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy129.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:118)
at org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:101)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:93)
at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:74)
at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:104)
at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-13 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 539 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 539 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/539/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 538 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 538 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/538/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 537 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 537 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/537/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 536 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 536 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/536/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 535 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 535 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/535/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 534 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 534 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/534/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 533 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 533 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/533/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 532 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 532 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/532/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 531 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 531 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/531/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 530 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 530 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/530/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 529 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 529 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/529/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 528 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 528 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/528/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 527 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 527 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/527/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 526 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 526 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/526/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 525 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 525 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/525/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 524 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 524 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/524/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 523 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 523 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/523/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 522 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 522 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/522/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 521 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 521 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/521/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 520 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 520 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/520/ to view the results.
Build failed in Jenkins: beam_LoadTests_Python_Combine_Dataflow_Streaming #519
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/519/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13653] Make SnsIO.write topicArn optional. If provided, validate
------------------------------------------
[...truncated 22.39 KB...]
Using cached azure_storage_blob-12.9.0-py2.py3-none-any.whl (356 kB)
Collecting azure-core>=1.7.0
Using cached azure_core-1.21.1-py2.py3-none-any.whl (178 kB)
Collecting freezegun>=0.3.12
Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting mock<3.0.0,>=1.0.1
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting parameterized<0.8.0,>=0.7.1
Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB)
Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9
Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB)
Collecting pyyaml<7.0.0,>=3.12
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Collecting requests_mock<2.0,>=1.7
Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB)
Collecting tenacity<6.0,>=5.0.2
Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB)
Collecting pytest<5.0,>=4.4.0
Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB)
Collecting pytest-xdist<2,>=1.29.0
Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB)
Collecting pytest-timeout<2,>=1.3.3
Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB)
Collecting sqlalchemy<2.0,>=1.3
Using cached SQLAlchemy-1.4.31-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting psycopg2-binary<3.0.0,>=2.8.5
Using cached psycopg2_binary-2.9.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting testcontainers<4.0.0,>=3.0.3
Using cached testcontainers-3.4.2-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: six>=1.11.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from azure-core>=1.7.0->apache-beam==2.37.0.dev0) (1.16.0)
Collecting msrest>=0.6.21
Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
Using cached cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.41
Using cached botocore-1.23.41-py3-none-any.whl (8.5 MB)
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting fasteners>=0.14
Using cached fasteners-0.17.2-py3-none-any.whl (18 kB)
Collecting rsa<5,>=3.1.4
Using cached rsa-4.8-py3-none-any.whl (39 kB)
Requirement already satisfied: setuptools>=40.3.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-auth<3,>=1.18.0->apache-beam==2.37.0.dev0) (60.5.0)
Collecting pyasn1-modules>=0.2.1
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-2.4.0-py2.py3-none-any.whl (111 kB)
Requirement already satisfied: packaging>=14.3 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.37.0.dev0) (21.3)
Collecting google-resumable-media<3.0dev,>=0.6.0
Using cached google_resumable_media-2.1.0-py2.py3-none-any.whl (75 kB)
Collecting libcst>=0.2.5
Using cached libcst-0.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.7 MB)
Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0
Using cached google_api_core-1.31.5-py2.py3-none-any.whl (93 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl
Collecting google-auth<3,>=1.18.0
Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting grpcio-status>=1.18.0
Using cached grpcio_status-1.43.0-py3-none-any.whl (10.0 kB)
Collecting overrides<7.0.0,>=6.0.1
Using cached overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting docopt
Using cached docopt-0.6.2-py2.py3-none-any.whl
Collecting pyparsing<3,>=2.4.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pbr>=0.11
Using cached pbr-5.8.0-py2.py3-none-any.whl (112 kB)
Collecting pyasn1>=0.1.7
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Requirement already satisfied: py>=1.5.0 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (1.11.0)
Collecting attrs>=17.4.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Requirement already satisfied: importlib-metadata>=0.12 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (2.1.3)
Collecting pluggy<1.0,>=0.12
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting more-itertools>=4.0.0
Using cached more_itertools-8.12.0-py3-none-any.whl (54 kB)
Collecting wcwidth
Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting atomicwrites>=1.0
Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting execnet>=1.1
Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting pytest-forked
Using cached pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
Using cached charset_normalizer-2.0.10-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting greenlet!=0.4.17
Using cached greenlet-1.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (150 kB)
Collecting deprecation
Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB)
Collecting wrapt
Using cached wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Collecting docker
Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)
Collecting cffi>=1.12
Using cached cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (427 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
Using cached googleapis_common_protos-1.54.0-py2.py3-none-any.whl (207 kB)
Collecting google-crc32c<2.0dev,>=1.0
Using cached google_crc32c-1.3.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38 kB)
Requirement already satisfied: zipp>=0.5 in /home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/build/gradleenv/1329484227/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.37.0.dev0) (3.7.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting pycparser
Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam: filename=apache_beam-2.37.0.dev0-py3-none-any.whl size=2645720 sha256=044ed37768654018fb30a7a729e1412c7e7cfcd67ee9ac96326ec7b269e4b749
Stored in directory: /home/jenkins/.cache/pip/wheels/e6/1f/28/9337974c607f5f016ef124646a645ebf2fb34bdd32df5f9dd8
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, typing-extensions, requests, pytz, mypy-extensions, googleapis-common-protos, google-auth, wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib, more-itertools, jmespath, google-api-core, attrs, atomicwrites, websocket-client, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst, isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, google-crc32c, docopt, cffi, botocore, wrapt, s3transfer, pytest-forked, pymongo, pydot, pyarrow, pbr, overrides, orjson, oauth2client, msrest, hdfs, grpcio-status, greenlet, google-resumable-media, google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, cryptography, crcmod, cloudpickle, azure-core, testcontainers, tenacity, sqlalchemy, requests-mock, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.37.0.dev0 atomicwrites-1.4.0 attrs-21.4.0 azure-core-1.21.1 azure-storage-blob-12.9.0 boto3-1.20.41 botocore-1.23.41 cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.10 cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.9 fasteners-0.17.2 freezegun-1.1.0 google-api-core-1.31.5 google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0 google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.5.0 google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0 google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.1.0 googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3 isodate-0.6.1 jmespath-0.10.0 libcst-0.4.0 mock-2.0.0 more-itertools-8.12.0 msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.21.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.6 overrides-6.1.0 pandas-1.3.5 parameterized-0.7.5 pbr-5.8.0 pluggy-0.13.1 proto-plus-1.19.8 psycopg2-binary-2.9.3 pyarrow-6.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.3 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3 pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.8 s3transfer-0.5.0 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2 typing-extensions-4.0.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8 wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :sdks:python:apache_beam:testing:load_tests:run
INFO:apache_beam.runners.portability.stager:Copying Beam SDK "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_Combine_Dataflow_Streaming/src/sdks/python/build/apache-beam.tar.gz" to staging location.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO:root:Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220117
INFO:root:Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/python37-fnapi:beam-master-20220117" for Docker environment
INFO:apache_beam.runners.dataflow.internal.apiclient:Defaulting to the temp_location as staging_location: gs://temp-storage-for-perf-tests/smoketests
INFO:apache_beam.internal.gcp.auth:Setting socket default timeout to 60 seconds.
INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0124150156.1643036846.881400/dataflow_python_sdk.tar...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0124150156.1643036846.881400/dataflow_python_sdk.tar in 0 seconds.
INFO:apache_beam.runners.dataflow.internal.apiclient:Starting GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0124150156.1643036846.881400/pipeline.pb...
INFO:apache_beam.runners.dataflow.internal.apiclient:Completed GCS upload to gs://temp-storage-for-perf-tests/smoketests/load-tests-python-dataflow-streaming-combine-1-0124150156.1643036846.881400/pipeline.pb in 0 seconds.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--top_count=20', 'shuffle_mode=appliance']
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--top_count=20', 'shuffle_mode=appliance']
INFO:apache_beam.runners.dataflow.internal.apiclient:Create job: <Job
clientRequestId: '20220124150726882441-8088'
createTime: '2022-01-24T15:07:28.142725Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2022-01-24_07_07_27-2924804929334068333'
location: 'us-central1'
name: 'load-tests-python-dataflow-streaming-combine-1-0124150156'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-01-24T15:07:28.142725Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
INFO:apache_beam.runners.dataflow.internal.apiclient:Created job with id: [2022-01-24_07_07_27-2924804929334068333]
INFO:apache_beam.runners.dataflow.internal.apiclient:Submitted job: 2022-01-24_07_07_27-2924804929334068333
INFO:apache_beam.runners.dataflow.internal.apiclient:To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-24_07_07_27-2924804929334068333?project=apache-beam-testing
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-24_07_07_27-2924804929334068333 is in state JOB_STATE_PENDING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:39.728Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.683Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.707Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.800Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.887Z: JOB_MESSAGE_DETAILED: Expanding SplittableProcessKeyed operations into optimizable parts.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.911Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into streaming Read/Write steps
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:40.976Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.047Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.092Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.125Z: JOB_MESSAGE_DETAILED: Fusing consumer Read synthetic/Map(<lambda at iobase.py:898>) into Read synthetic/Impulse
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.157Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction into Read synthetic/Map(<lambda at iobase.py:898>)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.187Z: JOB_MESSAGE_DETAILED: Fusing consumer ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/SplitWithSizing into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/PairWithRestriction
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.214Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: Start into ref_AppliedPTransform_Read-synthetic-SDFBoundedSourceReader-ParDo-SDFBoundedSourceDoFn-_6/ProcessElementAndRestrictionWithSizing
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.239Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/KeyWithVoid into Measure time: Start
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.273Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators into Combine with Top 0/KeyWithVoid
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.297Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/GroupByKey/WriteStream into Combine with Top 0/CombinePerKey/Combine/ConvertToAccumulators
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.321Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine into Combine with Top 0/CombinePerKey/GroupByKey/ReadStream
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.339Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/CombinePerKey/Combine/Extract into Combine with Top 0/CombinePerKey/Combine
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.372Z: JOB_MESSAGE_DETAILED: Fusing consumer Combine with Top 0/UnKey into Combine with Top 0/CombinePerKey/Combine/Extract
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.408Z: JOB_MESSAGE_DETAILED: Fusing consumer Consume 0 into Combine with Top 0/UnKey
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.429Z: JOB_MESSAGE_DETAILED: Fusing consumer Measure time: End 0 into Consume 0
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.524Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.559Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.594Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.637Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.704Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.741Z: JOB_MESSAGE_BASIC: Starting 5 ****s in us-central1-b...
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:41.818Z: JOB_MESSAGE_DEBUG: Starting **** pool setup.
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2022-01-24_07_07_27-2924804929334068333 is in state JOB_STATE_RUNNING
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:07:59.516Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:08:25.160Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:08:50.344Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:08:50.410Z: JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:oauth2client.transport:Refreshing due to a 401 (attempt 1/2)
INFO:apache_beam.runners.dataflow.dataflow_runner:2022-01-24T15:50:06.662Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Build timed out (after 120 minutes). Marking the build as aborted.
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@3437fe5b:apache-beam-jenkins-6": Remote call on apache-beam-jenkins-6 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:994)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy123.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException
at hudson.remoting.Channel.close(Channel.java:1499)
at hudson.remoting.Channel.close(Channel.java:1455)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:884)
at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:110)
at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:765)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-6 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 518 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 518 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/518/ to view the results.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 517 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_LoadTests_Python_Combine_Dataflow_Streaming - Build # 517 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_LoadTests_Python_Combine_Dataflow_Streaming/517/ to view the results.