You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2024/01/21 07:53:19 UTC
Build failed in Jenkins: beam_PostCommit_Python39 #2796
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2796/display/redirect>
Changes:
------------------------------------------
[...truncated 12.31 MB...]
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Sun, 21 Jan 2024 06:26:53 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
Worker ID: beamapp-jenkins-012106232-01202223-9f9u-harness-znv5
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-21T06:26:54.476Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0121062327-736512-zmwbf9ko.1705818207.736693/dax-tmp-2024-01-20_22_23_28-775354214300161484-S56-1-4ab3e737ccdbb1aa/tmp-4ab3e737ccdbb181@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-21T06:26:54.479Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0121062327-736512-zmwbf9ko.1705818207.736693/dax-tmp-2024-01-20_22_23_28-775354214300161484-S56-0-aae274e2907235b8/tmp-aae274e290723e73@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-21T06:26:54.483Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0121062327-736512-zmwbf9ko.1705818207.736693/dax-tmp-2024-01-20_22_23_28-775354214300161484-S56-2-6921924e8c0fbea3/tmp-6921924e8c0fb504@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-21T06:26:54.501Z: JOB_MESSAGE_WARNING: S56:WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write failed.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-21T06:26:54.525Z: JOB_MESSAGE_BASIC: Finished operation WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-21T06:26:54.894Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-21T06:28:57.451Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:151 Job 2024-01-20_22_23_28-775354214300161484 is in state JOB_STATE_FAILED
[1m[31mERROR [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:777 Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2024-01-20_22_23_28-775354214300161484?project=<ProjectId>
[32mINFO [0m apache_beam.io.gcp.bigquery_file_loads_test:bigquery_file_loads_test.py:1242 Deleting dataset python_bq_file_loads_1705818204a00455 in project apache-beam-testing
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery.py:2603
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2603: DeprecationWarning: invalid escape sequence \#
"""Read data from BigQuery.
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_batch_copy_jobs_with_no_input_schema[0m - AssertionError:
Expected: (<[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]>)
but: <[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]> was <DataflowPipelineResult 2024-01-20_22_16_25-16333248404773912906 DONE>
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_multiple_destinations_transform[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Sun, 21 Jan 2024 06:26:53 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Sun, 21 Jan 2024 06:26:53 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31m====== [31m[1m2 failed[0m, [32m89 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5832.29s (1:37:12)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 52
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 43m 28s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/k3gn2aicbzaxg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2807
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2807/display/redirect>
Changes:
------------------------------------------
[...truncated 8.22 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1706186479
nanos: 880798578
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:289"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706186479
nanos: 880876779
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/data_plane.py:820"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706186479
nanos: 880944490
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:929"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706186479
nanos: 881733655
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:301"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706186479
nanos: 881847381
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker_main.py:213"
thread: "MainThread"
173f01634ebf521caab901db0cd3e9fa79873f7ba492d7212cdb5cb64a070d8a
INFO:apache_beam.runners.portability.local_job_service:Completed job in 35.60045385360718 seconds with state DONE.
INFO:root:Completed job in 35.60045385360718 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py39:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36093
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7b34cac5e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7b34cac670> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7b34cacdc0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ('java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-2.55.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempotve5sha/artifactsdty6jhl4' '--job-port' '58783' '--artifact-port' '0' '--expansion-port' '0')
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:31 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36667
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35683
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:58783
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C
WARNING:root:Waiting for grpc channel to be ready at localhost:58783.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_33fe9a6b-2c58-4d77-b9db-5c6de6bb207c.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_33fe9a6b-2c58-4d77-b9db-5c6de6bb207c.0:ref_Environment_default_environment_1.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_33fe9a6b-2c58-4d77-b9db-5c6de6bb207c.null.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_33fe9a6b-2c58-4d77-b9db-5c6de6bb207c.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:34 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0125124134-a50b9acb_cb328bc3-63a4-4072-a60b-e3414e3a4810
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:34 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0125124134-a50b9acb_cb328bc3-63a4-4072-a60b-e3414e3a4810
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:34 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:35 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.util.log: Logging initialized @8780ms to org.sparkproject.jetty.util.log.Slf4jLog
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.Server: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 1.8.0_392-8u392-ga-1~20.04-b08
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.Server: Started @8943ms
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.AbstractConnector: Started ServerConnector@3dae3c01{HTTP/1.1, (http/1.1)}{127.0.0.1:4040}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@512e655d{/jobs,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3dc976a3{/jobs/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@16e343da{/jobs/job,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3d670694{/jobs/job/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bcbc4e3{/stages,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1270d0ae{/stages/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@55b66a4a{/stages/stage,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6bd836b4{/stages/stage/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@75fc2c80{/stages/pool,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5f04c85{/stages/pool/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2ae758e3{/storage,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3fc3252f{/storage/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4e25054f{/storage/rdd,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@569191c4{/storage/rdd/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@45e90764{/environment,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6ad625c{/environment/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2b5bdbfd{/executors,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5c465729{/executors/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2e67edb9{/executors/threadDump,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5296a9c2{/executors/threadDump/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2a7afc9c{/static,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4800bb91{/,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5228f209{/api,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@363f9a5b{/jobs/job/kill,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7b6836ec{/stages/stage/kill,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@67cb8fe9{/metrics/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:37 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0125124134-a50b9acb_cb328bc3-63a4-4072-a60b-e3414e3a4810 on Spark master local[4]
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 104857600
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39113.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36623.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40839
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0125124134-a50b9acb_cb328bc3-63a4-4072-a60b-e3414e3a4810: Pipeline translated successfully. Computing outputs
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:41 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0125124134-a50b9acb_cb328bc3-63a4-4072-a60b-e3414e3a4810 finished.
INFO:apache_beam.utils.subprocess_server:24/01/25 12:41:41 INFO org.sparkproject.jetty.server.AbstractConnector: Stopped Spark@3dae3c01{HTTP/1.1, (http/1.1)}{127.0.0.1:4040}
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:40839 {created_time:"2024-01-25T12:41:41.662942116+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self.run()
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 266, in run
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1060, in pull_responses
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
self._target(*self._args, **self._kwargs)
return self._next()
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
target=lambda: self._read_inputs(elements_iterator),
raise self
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:39113 {created_time:"2024-01-25T12:41:41.662982271+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:36623 {created_time:"2024-01-25T12:41:41.66297486+00:00", grpc_status:14, grpc_message:"Socket closed"}"
>
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:40839 {created_time:"2024-01-25T12:41:41.662942116+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>
> Task :sdks:python:test-suites:portable:py39:postCommitPy39
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/-1734967050/bin/python3.9>: No module named build
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 43m 36s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/ojxvfojaun23m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2806
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2806/display/redirect>
Changes:
------------------------------------------
[...truncated 8.37 MB...]
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 06:29:12 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 06:29:12 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31mFAILED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::[1mHourlyTeamScoreIT::test_hourly_team_score_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 07:37:25 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 07:37:25 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-80']
[31m====== [31m[1m3 failed[0m, [32m89 passed[0m, [33m51 skipped[0m, [33m8 warnings[0m[31m in 5675.96s (1:34:35)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 41m 33s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/aqdytsifdmwca
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2805
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2805/display/redirect>
Changes:
------------------------------------------
[...truncated 12.37 MB...]
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 00:29:10 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
Worker ID: beamapp-jenkins-012500254-01241625-uy51-harness-w2wg
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-25T00:29:11.982Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0125002540-066487-zmwbf9ko.1706142340.066646/dax-tmp-2024-01-24_16_25_40-3576056548624936316-S56-2-ba1d8146837ee981/tmp-ba1d8146837eed58@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-25T00:29:11.983Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0125002540-066487-zmwbf9ko.1706142340.066646/dax-tmp-2024-01-24_16_25_40-3576056548624936316-S56-1-a88f64f614a2a437/tmp-a88f64f614a2a2e2@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-25T00:29:11.987Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0125002540-066487-zmwbf9ko.1706142340.066646/dax-tmp-2024-01-24_16_25_40-3576056548624936316-S56-0-aa9e464cd130d213/tmp-aa9e464cd130d6be@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-25T00:29:12.006Z: JOB_MESSAGE_WARNING: S56:WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write failed.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-25T00:29:12.031Z: JOB_MESSAGE_BASIC: Finished operation WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-25T00:29:12.427Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-25T00:31:37.231Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:151 Job 2024-01-24_16_25_40-3576056548624936316 is in state JOB_STATE_FAILED
[1m[31mERROR [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:777 Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2024-01-24_16_25_40-3576056548624936316?project=<ProjectId>
[32mINFO [0m apache_beam.io.gcp.bigquery_file_loads_test:bigquery_file_loads_test.py:1242 Deleting dataset python_bq_file_loads_170614233681bd22 in project apache-beam-testing
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery.py:2634
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2634: DeprecationWarning: invalid escape sequence \#
"""Read data from BigQuery.
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_batch_copy_jobs_with_no_input_schema[0m - AssertionError:
Expected: (<[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]>)
but: <[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]> was <DataflowPipelineResult 2024-01-24_16_18_12-12488496925338720336 DONE>
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_multiple_destinations_transform[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 00:29:10 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 25 Jan 2024 00:29:10 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31m====== [31m[1m2 failed[0m, [32m90 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5741.10s (1:35:41)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 52
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 42m 5s
222 actionable tasks: 158 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/3drzg7ginkvdq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2804
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2804/display/redirect>
Changes:
------------------------------------------
[...truncated 12.52 MB...]
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 18:30:39 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 18:30:39 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31mFAILED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::[1mHourlyTeamScoreIT::test_hourly_team_score_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 19:32:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 19:32:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-80']
[31m====== [31m[1m3 failed[0m, [32m88 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5913.90s (1:38:33)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 46m 29s
222 actionable tasks: 205 executed, 13 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/7nh46aey2e5rw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2803
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2803/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #30059 [YAML] Require numStreams for unbounded
------------------------------------------
[...truncated 11.83 MB...]
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 12:28:50 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
Worker ID: beamapp-jenkins-012412254-01240425-37cr-harness-zvln
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-24T12:28:51.696Z: JOB_MESSAGE_BASIC: Finished operation WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-24T12:28:52.543Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-24T12:30:59.375Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:151 Job 2024-01-24_04_25_43-14134279486379602501 is in state JOB_STATE_FAILED
[31m[1mERROR [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:777 Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2024-01-24_04_25_43-14134279486379602501?project=<ProjectId>
[32mINFO [0m apache_beam.io.gcp.bigquery_file_loads_test:bigquery_file_loads_test.py:1242 Deleting dataset python_bq_file_loads_170609913902b7e5 in project apache-beam-testing
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery.py:2603
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2603: DeprecationWarning: invalid escape sequence \#
"""Read data from BigQuery.
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_batch_copy_jobs_with_no_input_schema[0m - AssertionError:
Expected: (<[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]>)
but: <[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]> was <DataflowPipelineResult 2024-01-24_04_17_54-14256806591439246841 DONE>
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_multiple_destinations_transform[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 12:28:50 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 12:28:50 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31m====== [31m[1m2 failed[0m, [32m89 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5872.80s (1:37:52)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 44m 1s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/i46l7mggxo7is
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2802
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2802/display/redirect>
Changes:
------------------------------------------
[...truncated 12.43 MB...]
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 06:28:22 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31mFAILED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::[1mHourlyTeamScoreIT::test_hourly_team_score_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 07:25:37 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 24 Jan 2024 07:25:37 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-80']
[31m====== [31m[1m3 failed[0m, [32m88 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5672.83s (1:34:32)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 52
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
==============================================================================
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 40m 51s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/ofnztmax6pijq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2801
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2801/display/redirect>
Changes:
------------------------------------------
[...truncated 10.89 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.grpc.v1p60p1.com.google.common.base.Preconditions.checkState(Preconditions.java:512)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.grpc.v1p60p1.io.grpc.internal.ServerCallImpl.closeInternal(ServerCallImpl.java:213)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.grpc.v1p60p1.io.grpc.internal.ServerCallImpl.close(ServerCallImpl.java:208)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.grpc.v1p60p1.io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted(ServerCalls.java:395)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.state.GrpcStateService.close(GrpcStateService.java:63)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.sdk.fn.server.GrpcFnServer.close(GrpcFnServer.java:156)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:649)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:649)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:665)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$400(DefaultJobBundleFactory.java:596)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCaches$3(DefaultJobBundleFactory.java:213)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1850)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3503)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3479)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3252)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4343)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4960)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:326)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:43)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:217)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:193)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:182)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.scheduleRelease(ReferenceCountingExecutableStageContextFactory.java:136)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.access$300(ReferenceCountingExecutableStageContextFactory.java:48)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.close(ReferenceCountingExecutableStageContextFactory.java:213)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.flink.translation.wrappers.streaming.ExecutableStageDoFnOperator.$closeResource(ExecutableStageDoFnOperator.java:550)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.flink.translation.wrappers.streaming.ExecutableStageDoFnOperator.cleanUp(ExecutableStageDoFnOperator.java:743)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.beam.runners.flink.translation.wrappers.streaming.AbstractStreamOperatorCompat.close(AbstractStreamOperatorCompat.java:86)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:163)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.closeAllOperators(RegularOperatorChain.java:125)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.streaming.runtime.tasks.StreamTask.closeAllOperators(StreamTask.java:1025)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:903)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:786)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 at java.lang.Thread.run(Thread.java:750)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Jan 24, 2024 12:42:11 AM org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient unref
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 INFO: Closing environment urn: "beam:env:docker:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 payload: "\n apache/beam_java8_sdk:2.54.0.dev"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:bytes:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:bool:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:varint:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:string_utf8:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:iterable:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:timer:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:kv:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:length_prefix:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:global_window:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:interval_window:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:custom_window:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:windowed_value:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:double:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:row:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:param_windowed_value:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:state_backed_iterable:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:sharded_key:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:coder:nullable:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:multi_core_bundle_processing:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:progress_reporting:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:harness_monitoring_infos:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:control_request_elements_embedding:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:state_caching:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:version:sdk_base:apache/beam_java8_sdk:2.54.0.dev"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:transform:to_string:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 capabilities: "beam:protocol:data_sampling:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 dependencies {
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 type_urn: "beam:artifact:type:file:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 type_payload: "\n\316\001/tmp/beam-tempz5_enx_b/artifactsov9_5wt0/9cdb454fa7be137118b35b8968a45585f12270d75eb0a489b012ee06faae8be5/1-0:external_12beam:env:any-beam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJ"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 role_urn: "beam:artifact:role:staging_to:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 role_payload: "\nlbeam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJEwfIY80Ghh-hoTQuq0gCY0_cfQWHOD4Q.jar"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 }
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Jan 24, 2024 12:42:11 AM org.apache.beam.runners.fnexecution.logging.GrpcLoggingService close
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 INFO: 2 Beam Fn Logging clients still connected during shutdown.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Jan 24, 2024 12:42:11 AM org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver onCompleted
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 INFO: Hanged up for unknown endpoint.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Jan 24, 2024 12:42:11 AM org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment close
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 INFO: Closing Docker container 7c14ca6688ad875f11676029c715b099003c16f198dd1b578ed654c8107d806d. Logs:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 2024/01/24 00:42:05 Provision info:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 pipeline_options:{fields:{key:"beam:option:allow_non_deterministic_key_coders:v1" value:{bool_value:false}} fields:{key:"beam:option:allow_non_restored_state:v1" value:{bool_value:false}} fields:{key:"beam:option:allow_unsafe_triggers:v1" value:{bool_value:false}} fields:{key:"beam:option:app_name:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:artifact_port:v1" value:{string_value:"0"}} fields:{key:"beam:option:attached_mode:v1" value:{bool_value:true}} fields:{key:"beam:option:auto_balance_write_files_sharding_enabled:v1" value:{bool_value:false}} fields:{key:"beam:option:auto_unique_labels:v1" value:{bool_value:false}} fields:{key:"beam:option:beam_services:v1" value:{struct_value:{}}} fields:{key:"beam:option:block_on_run:v1" value:{bool_value:true}} fields:{key:"beam:option:dataflow_endpoint:v1" value:{string_value:"https://dataflow.googleapis.com"}} fields:{key:"beam:option:direct_embed_docker_python:v1" value:{bool_value:false}} fields:{key:"beam:option:direct_num_workers:v1" value:{string_value:"1"}} fields:{key:"beam:option:direct_runner_bundle_repeat:v1" value:{string_value:"0"}} fields:{key:"beam:option:direct_runner_use_stacked_bundle:v1" value:{bool_value:true}} fields:{key:"beam:option:direct_running_mode:v1" value:{string_value:"in_memory"}} fields:{key:"beam:option:direct_test_splits:v1" value:{struct_value:{}}} fields:{key:"beam:option:disable_metrics:v1" value:{bool_value:false}} fields:{key:"beam:option:dry_run:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_artifact_caching:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_bundling:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_heap_dumps:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_hot_key_logging:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_stable_input_drain:v1" value:{bool_value:false}} fields:{key:"beam:option:enable_streaming_engine:v1" value:{bool_value:false}} fields:{key:"beam:option:enforce_encodability:v1" value:{bool_value:true}} fields:{key:"beam:option:enforce_immutability:v1" value:{bool_value:true}} fields:{key:"beam:option:environment_cache_millis:v1" value:{string_value:"0"}} fields:{key:"beam:option:environment_type:v1" value:{string_value:"LOOPBACK"}} fields:{key:"beam:option:expansion_port:v1" value:{string_value:"0"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:externalized_checkpoints_enabled:v1" value:{bool_value:false}} fields:{key:"beam:option:fail_on_checkpointing_errors:v1" value:{bool_value:true}} fields:{key:"beam:option:faster_copy:v1" value:{bool_value:false}} fields:{key:"beam:option:finish_bundle_before_checkpointing:v1" value:{bool_value:false}} fields:{key:"beam:option:flink_conf_dir:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:flink_job_server_jar:v1" value:{string_value:"<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/runners/flink/1.16/job-server/build/libs/beam-runners-flink-1.16-job-server-2.54.0-SNAPSHOT.jar"}}> fields:{key:"beam:option:flink_master:v1" value:{string_value:"[auto]"}} fields:{key:"beam:option:flink_submit_uber_jar:v1" value:{bool_value:false}} fields:{key:"beam:option:flink_version:v1" value:{string_value:"1.16"}} fields:{key:"beam:option:gcp_oauth_scopes:v1" value:{list_value:{values:{string_value:"https://www.googleapis.com/auth/bigquery"} values:{string_value:"https://www.googleapis.com/auth/cloud-platform"} values:{string_value:"https://www.googleapis.com/auth/devstorage.full_control"} values:{string_value:"https://www.googleapis.com/auth/userinfo.email"} values:{string_value:"https://www.googleapis.com/auth/datastore"} values:{string_value:"https://www.googleapis.com/auth/spanner.admin"} values:{string_value:"https://www.googleapis.com/auth/spanner.data"}}}} fields:{key:"beam:option:gcs_performance_metrics:v1" value:{bool_value:false}} fields:{key:"beam:option:hdfs_full_urls:v1" value:{bool_value:false}} fields:{key:"beam:option:job_name:v1" value:{string_value:"BeamApp-jenkins-0124004158-75a26fd0"}} fields:{key:"beam:option:job_port:v1" value:{string_value:"0"}} fields:{key:"beam:option:job_server_java_launcher:v1" value:{string_value:"java"}} fields:{key:"beam:option:job_server_jvm_properties:v1" value:{list_value:{}}} fields:{key:"beam:option:job_server_timeout:v1" value:{string_value:"60"}} fields:{key:"beam:option:load_balance_bundles:v1" value:{bool_value:false}} fields:{key:"beam:option:log_mdc:v1" value:{bool_value:true}} fields:{key:"beam:option:max_cache_memory_usage_mb:v1" value:{string_value:"100"}} fields:{key:"beam:option:max_parallelism:v1" value:{string_value:"-1"}} fields:{key:"beam:option:no_auth:v1" value:{bool_value:false}} fields:{key:"beam:option:object_reuse:v1" value:{bool_value:false}} fields:{key:"beam:option:operator_chaining:v1" value:{bool_value:true}} fields:{key:"beam:option:options_id:v1" value:{number_value:2}} fields:{key:"beam:option:output_executable_path:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:parallelism:v1" value:{string_value:"-1"}} fields:{key:"beam:option:performance_runtime_type_check:v1" value:{bool_value:false}} fields:{key:"beam:option:pickle_library:v1" value:{string_value:"default"}} fields:{key:"beam:option:pipeline_type_check:v1" value:{bool_value:true}} fields:{key:"beam:option:profile_cpu:v1" value:{bool_value:false}} fields:{key:"beam:option:profile_memory:v1" value:{bool_value:false}} fields:{key:"beam:option:profile_sample_rate:v1" value:{number_value:1}} fields:{key:"beam:option:project:v1" value:{string_value:"apache-beam-testing"}} fields:{key:"beam:option:re_iterable_group_by_key_result:v1" value:{bool_value:false}} fields:{key:"beam:option:requirements_cache_only_sources:v1" value:{bool_value:false}} fields:{key:"beam:option:resource_hints:v1" value:{list_value:{}}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:retain_externalized_checkpoints_on_cancellation:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:runner_determined_sharding:v1" value:{bool_value:true}} fields:{key:"beam:option:runtime_type_check:v1" value:{bool_value:false}} fields:{key:"beam:option:s3_disable_ssl:v1" value:{bool_value:false}} fields:{key:"beam:option:save_main_session:v1" value:{bool_value:false}} fields:{key:"beam:option:sdk_harness_log_level_overrides:v1" value:{struct_value:{fields:{key:"org.apache.beam.runners.flink.FlinkPipelineRunner" value:{string_value:"WARN"}} fields:{key:"org.apache.flink.runtime" value:{string_value:"WARN"}} fields:{key:"org.apache.flink.runtime.metrics.groups" value:{string_value:"ERROR"}}}}} fields:{key:"beam:option:sdk_location:v1" value:{string_value:"container"}} fields:{key:"beam:option:sdk_worker_parallelism:v1" value:{string_value:"1"}} fields:{key:"beam:option:spark_master_url:v1" value:{string_value:"local[4]"}} fields:{key:"beam:option:spark_submit_uber_jar:v1" value:{bool_value:false}} fields:{key:"beam:option:spark_version:v1" value:{string_value:"3"}} fields:{key:"beam:option:streaming:v1" value:{bool_value:false}} fields:{key:"beam:option:temp_location:v1" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-it"}} fields:{key:"beam:option:type_check_additional:v1" value:{string_value:""}} fields:{key:"beam:option:type_check_strictness:v1" value:{string_value:"DEFAULT_TO_ANY"}} fields:{key:"beam:option:update:v1" value:{bool_value:false}} fields:{key:"beam:option:use_data_stream_for_batch:v1" value:{bool_value:false}} fields:{key:"beam:option:use_storage_api_connection_pool:v1" value:{bool_value:false}} fields:{key:"beam:option:use_storage_write_api:v1" value:{bool_value:false}} fields:{key:"beam:option:use_storage_write_api_at_least_once:v1" value:{bool_value:false}} fields:{key:"beam:option:use_transform_service:v1" value:{bool_value:false}} fields:{key:"beam:option:verify_row_values:v1" value:{bool_value:false}}} logging_endpoint:{url:"localhost:36491"} artifact_endpoint:{url:"localhost:40661"} control_endpoint:{url:"localhost:36887"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\xce\x01/tmp/beam-tempz5_enx_b/artifactsov9_5wt0/9cdb454fa7be137118b35b8968a45585f12270d75eb0a489b012ee06faae8be5/1-0:external_12beam:env:any-beam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJ" role_urn:"beam:artifact:role:staging_to:v1" role_payload:"\nlbeam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJEwfIY80Ghh-hoTQuq0gCY0_cfQWHOD4Q.jar"} runner_capabilities:"beam:protocol:control_response_elements_embedding:v1"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 2024/01/24 00:42:06 Downloaded: /tmp/1-2/staged/beam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJEwfIY80Ghh-hoTQuq0gCY0_cfQWHOD4Q.jar (sha256: 33feb2a29a6cd23244c1f218f341a187e8684d0baad20098d3f71f4161ce0f84, size: 78791353)
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SLF4J: Class path contains multiple SLF4J bindings.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SLF4J: Found binding in [jar:file:/opt/apache/beam/jars/slf4j-jdk14.jar!/org/slf4j/impl/StaticLoggerBinder.class]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SLF4J: Found binding in [jar:file:/tmp/1-2/staged/beam-sdks-java-io-debezium-expansion-service-2.54.0-SNAPSHOT-M_6yopps0jJEwfIY80Ghh-hoTQuq0gCY0_cfQWHOD4Q.jar!/org/slf4j/impl/StaticLoggerBinder.class]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 SDK Fn Harness started
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Harness ID 1-2
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Logging location url: "localhost:36491"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Control location url: "localhost:36887"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Status location null
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Pipeline Options File pipeline_options.json
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Pipeline Options File pipeline_options.json exists. Overriding existing options.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Pipeline options {"beam:option:allow_non_deterministic_key_coders:v1":false,"beam:option:allow_non_restored_state:v1":false,"beam:option:allow_unsafe_triggers:v1":false,"beam:option:app_name:v1":null,"beam:option:artifact_port:v1":"0","beam:option:attached_mode:v1":true,"beam:option:auto_balance_write_files_sharding_enabled:v1":false,"beam:option:auto_unique_labels:v1":false,"beam:option:beam_services:v1":{},"beam:option:block_on_run:v1":true,"beam:option:dataflow_endpoint:v1":"https://dataflow.googleapis.com","beam:option:direct_embed_docker_python:v1":false,"beam:option:direct_num_workers:v1":"1","beam:option:direct_runner_bundle_repeat:v1":"0","beam:option:direct_runner_use_stacked_bundle:v1":true,"beam:option:direct_running_mode:v1":"in_memory","beam:option:direct_test_splits:v1":{},"beam:option:disable_metrics:v1":false,"beam:option:dry_run:v1":false,"beam:option:enable_artifact_caching:v1":false,"beam:option:enable_bundling:v1":false,"beam:option:enable_heap_dumps:v1":false,"beam:option:enable_hot_key_logging:v1":false,"beam:option:enable_stable_input_drain:v1":false,"beam:option:enable_streaming_engine:v1":false,"beam:option:enforce_encodability:v1":true,"beam:option:enforce_immutability:v1":true,"beam:option:environment_cache_millis:v1":"0","beam:option:environment_type:v1":"LOOPBACK","beam:option:expansion_port:v1":"0","beam:option:experiments:v1":["beam_fn_api"],"beam:option:externalized_checkpoints_enabled:v1":false,"beam:option:fail_on_checkpointing_errors:v1":true,"beam:option:faster_copy:v1":false,"beam:option:finish_bundle_before_checkpointing:v1":false,"beam:option:flink_conf_dir:v1":null,"beam:option:flink_job_server_jar:v1":"<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/runners/flink/1.16/job-server/build/libs/beam-runners-flink-1.16-job-server-2.54.0-SNAPSHOT.jar","beam>:option:flink_master:v1":"[auto]","beam:option:flink_submit_uber_jar:v1":false,"beam:option:flink_version:v1":"1.16","beam:option:gcp_oauth_scopes:v1":["https://www.googleapis.com/auth/bigquery","https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/devstorage.full_control","https://www.googleapis.com/auth/userinfo.email","https://www.googleapis.com/auth/datastore","https://www.googleapis.com/auth/spanner.admin","https://www.googleapis.com/auth/spanner.data"],"beam:option:gcs_performance_metrics:v1":false,"beam:option:hdfs_full_urls:v1":false,"beam:option:job_name:v1":"BeamApp-jenkins-0124004158-75a26fd0","beam:option:job_port:v1":"0","beam:option:job_server_java_launcher:v1":"java","beam:option:job_server_jvm_properties:v1":[],"beam:option:job_server_timeout:v1":"60","beam:option:load_balance_bundles:v1":false,"beam:option:log_mdc:v1":true,"beam:option:max_cache_memory_usage_mb:v1":"100","beam:option:max_parallelism:v1":"-1","beam:option:no_auth:v1":false,"beam:option:object_reuse:v1":false,"beam:option:operator_chaining:v1":true,"beam:option:options_id:v1":2,"beam:option:output_executable_path:v1":null,"beam:option:parallelism:v1":"-1","beam:option:performance_runtime_type_check:v1":false,"beam:option:pickle_library:v1":"default","beam:option:pipeline_type_check:v1":true,"beam:option:profile_cpu:v1":false,"beam:option:profile_memory:v1":false,"beam:option:profile_sample_rate:v1":1,"beam:option:project:v1":"apache-beam-testing","beam:option:re_iterable_group_by_key_result:v1":false,"beam:option:requirements_cache_only_sources:v1":false,"beam:option:resource_hints:v1":[],"beam:option:retain_docker_containers:v1":false,"beam:option:retain_externalized_checkpoints_on_cancellation:v1":false,"beam:option:runner:v1":null,"beam:option:runner_determined_sharding:v1":true,"beam:option:runtime_type_check:v1":false,"beam:option:s3_disable_ssl:v1":false,"beam:option:save_main_session:v1":false,"beam:option:sdk_harness_log_level_overrides:v1":{"org.apache.beam.runners.flink.FlinkPipelineRunner":"WARN","org.apache.flink.runtime":"WARN","org.apache.flink.runtime.metrics.groups":"ERROR"},"beam:option:sdk_location:v1":"container","beam:option:sdk_worker_parallelism:v1":"1","beam:option:spark_master_url:v1":"local[4]","beam:option:spark_submit_uber_jar:v1":false,"beam:option:spark_version:v1":"3","beam:option:streaming:v1":false,"beam:option:temp_location:v1":"gs://temp-storage-for-end-to-end-tests/temp-it","beam:option:type_check_additional:v1":"","beam:option:type_check_strictness:v1":"DEFAULT_TO_ANY","beam:option:update:v1":false,"beam:option:use_data_stream_for_batch:v1":false,"beam:option:use_storage_api_connection_pool:v1":false,"beam:option:use_storage_write_api:v1":false,"beam:option:use_storage_write_api_at_least_once:v1":false,"beam:option:use_transform_service:v1":false,"beam:option:verify_row_values:v1":false}
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:550 Job state changed to DONE
PASSED [100%]
=============================== warnings summary ===============================
apache_beam/io/external/xlang_jdbcio_it_test.py::CrossLanguageJdbcIOTest::test_xlang_jdbc_write_read_0_postgres
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/external/xlang_jdbcio_it_test.py>:132: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
self.engine.execute(
apache_beam/io/external/xlang_jdbcio_it_test.py::CrossLanguageJdbcIOTest::test_xlang_jdbc_write_read_0_postgres
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/_pytest/threadexception.py>:73: PytestUnhandledThreadExceptionWarning: Exception in thread read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:45747 {grpc_message:"Multiplexer hanging up", grpc_status:1, created_time:"2024-01-24T00:37:47.069406675+00:00"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
apache_beam/io/external/xlang_jdbcio_it_test.py::CrossLanguageJdbcIOTest::test_xlang_jdbc_write_read_1_mysql
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/_pytest/threadexception.py>:73: PytestUnhandledThreadExceptionWarning: Exception in thread read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:44021 {grpc_message:"Multiplexer hanging up", grpc_status:1, created_time:"2024-01-24T00:39:28.547541461+00:00"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
apache_beam/io/external/xlang_kafkaio_it_test.py::CrossLanguageKafkaIOTest::test_local_kafkaio_null_key
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/_pytest/threadexception.py>:73: PytestUnhandledThreadExceptionWarning: Exception in thread read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:38407 {created_time:"2024-01-24T00:40:14.838872697+00:00", grpc_status:1, grpc_message:"Multiplexer hanging up"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
apache_beam/io/external/xlang_kinesisio_it_test.py::CrossLanguageKinesisIOTest::test_kinesis_write
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/_pytest/threadexception.py>:73: PytestUnhandledThreadExceptionWarning: Exception in thread read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:43439 {grpc_message:"Multiplexer hanging up", grpc_status:1, created_time:"2024-01-24T00:41:40.719421118+00:00"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-flink-py39.xml> -
============ 20 passed, 7 skipped, 5 warnings in 656.76s (0:10:56) =============
> Task :sdks:python:test-suites:portable:py39:postCommitPy39
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/-1734967050/bin/python3.9>: No module named build
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 47m 15s
222 actionable tasks: 197 executed, 21 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/du6l3cmyapltg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2800
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2800/display/redirect>
Changes:
------------------------------------------
[...truncated 11.72 MB...]
seconds: 1706035138
nanos: 662112951
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:289"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706035138
nanos: 662215471
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/data_plane.py:820"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706035138
nanos: 662307977
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:929"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706035138
nanos: 662584066
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py:301"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1706035138
nanos: 662683725
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker_main.py:213"
thread: "MainThread"
c3947c3066ab8959aff1bf6908d8567c0b859a42a285bb5013f6719897d54a20
INFO:apache_beam.runners.portability.local_job_service:Completed job in 28.019731044769287 seconds with state DONE.
INFO:root:Completed job in 28.019731044769287 seconds with state DONE.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py39:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34511
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f91912f5310> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f91912f53a0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f91912f5af0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ('java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-2.54.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempcawayi0i/artifactscibtmg17' '--job-port' '32905' '--artifact-port' '0' '--expansion-port' '0')
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:10 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
WARNING:root:Waiting for grpc channel to be ready at localhost:32905.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34513
WARNING:root:Waiting for grpc channel to be ready at localhost:32905.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46507
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:32905
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:12 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C
WARNING:root:Waiting for grpc channel to be ready at localhost:32905.
WARNING:root:Waiting for grpc channel to be ready at localhost:32905.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d5dcf937-e2e0-4dde-9979-0c5c19bea27f.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d5dcf937-e2e0-4dde-9979-0c5c19bea27f.0:ref_Environment_default_environment_1.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d5dcf937-e2e0-4dde-9979-0c5c19bea27f.null.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:15 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d5dcf937-e2e0-4dde-9979-0c5c19bea27f.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:16 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-0123183916-cd540e27_cc9952f6-b42d-4a51-9cc3-1c5008c9ab54
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:16 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-0123183916-cd540e27_cc9952f6-b42d-4a51-9cc3-1c5008c9ab54
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:16 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:17 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.util.log: Logging initialized @13127ms to org.sparkproject.jetty.util.log.Slf4jLog
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.Server: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 1.8.0_392-8u392-ga-1~20.04-b08
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.Server: Started @13287ms
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.AbstractConnector: Started ServerConnector@63ca876b{HTTP/1.1, (http/1.1)}{127.0.0.1:4040}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3f9906cd{/jobs,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e370134{/jobs/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2e28800e{/jobs/job,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@51292f63{/jobs/job/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@43c75f6d{/stages,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@51aac9dd{/stages/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4c8de59e{/stages/stage,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@56145873{/stages/stage/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4fc13502{/stages/pool,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@79248c91{/stages/pool/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@49190559{/storage,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@25e69ee8{/storage/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@191b0d45{/storage/rdd,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2e6dc31a{/storage/rdd/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@38a5e291{/environment,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2ff4be36{/environment/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6cdd85be{/executors,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7dc801d{/executors/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@267f6247{/executors/threadDump,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@54d6b16d{/executors/threadDump/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@50aa139f{/static,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6a3da255{/,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1a9c6157{/api,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@63563dae{/jobs/job/kill,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:19 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@46085c71{/stages/stage/kill,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:20 INFO org.sparkproject.jetty.server.handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6fd6b4e1{/metrics/json,null,AVAILABLE,@Spark}
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:20 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:20 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-0123183916-cd540e27_cc9952f6-b42d-4a51-9cc3-1c5008c9ab54 on Spark master local[4]
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 104857600
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38673.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35511.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46305
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0123183916-cd540e27_cc9952f6-b42d-4a51-9cc3-1c5008c9ab54: Pipeline translated successfully. Computing outputs
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.01 seconds.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0123183916-cd540e27_cc9952f6-b42d-4a51-9cc3-1c5008c9ab54 finished.
INFO:apache_beam.utils.subprocess_server:24/01/23 18:39:23 INFO org.sparkproject.jetty.server.AbstractConnector: Stopped Spark@63ca876b{HTTP/1.1, (http/1.1)}{127.0.0.1:4040}
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 266, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 1060, in pull_responses
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:46305 {created_time:"2024-01-23T18:39:24.375459668+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
raise self
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:35511 {created_time:"2024-01-23T18:39:24.375469193+00:00", grpc_status:14, grpc_message:"Socket closed"}"
>
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:38673 {grpc_message:"recvmsg:Connection reset by peer", grpc_status:14, created_time:"2024-01-23T18:39:24.375476938+00:00"}"
>
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 686, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 669, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 540, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/2022703443/lib/python3.9/site-packages/grpc/_channel.py",> line 966, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "recvmsg:Connection reset by peer"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:46305 {created_time:"2024-01-23T18:39:24.375459668+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>
> Task :sdks:python:test-suites:portable:py39:postCommitPy39
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/-1734967050/bin/python3.9>: No module named build
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 43m 21s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/7lz7sozwpjxgs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2799
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2799/display/redirect>
Changes:
------------------------------------------
[...truncated 12.56 MB...]
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 23 Jan 2024 16:03:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 23 Jan 2024 16:03:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31mFAILED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::[1mHourlyTeamScoreIT::test_hourly_team_score_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 23 Jan 2024 17:14:22 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 23 Jan 2024 17:14:22 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-80']
[31m====== [31m[1m3 failed[0m, [32m88 passed[0m, [33m51 skipped[0m, [33m8 warnings[0m[31m in 6831.09s (1:53:51)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 2h 6m 24s
222 actionable tasks: 160 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/clm6peiguwnz6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2798
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2798/display/redirect?page=changes>
Changes:
[noreply] split write rename and temp table deletion into to dofns with a shuffle
------------------------------------------
[...truncated 11.90 MB...]
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 23:32:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
Worker ID: beamapp-jenkins-012223285-01221528-xc4o-harness-ncnc
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-22T23:32:19.789Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0122232858-568015-zmwbf9ko.1705966138.568187/dax-tmp-2024-01-22_15_28_59-17156436973935848627-S56-1-19dcc1887875aa01/tmp-19dcc1887875aa3e@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-22T23:32:19.790Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0122232858-568015-zmwbf9ko.1705966138.568187/dax-tmp-2024-01-22_15_28_59-17156436973935848627-S56-0-21823c6311841284/tmp-21823c6311841217@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-22T23:32:19.799Z: JOB_MESSAGE_WARNING: Unable to delete temp files: "gs://temp-storage-for-end-to-end-tests/temp-it/beamapp-jenkins-0122232858-568015-zmwbf9ko.1705966138.568187/dax-tmp-2024-01-22_15_28_59-17156436973935848627-S56-2-2f1d67533a412e60/tmp-2f1d67533a412c19@DAX.recordio."
[33mWARNING [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:203 2024-01-22T23:32:19.819Z: JOB_MESSAGE_WARNING: S56:WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write failed.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-22T23:32:19.842Z: JOB_MESSAGE_BASIC: Finished operation WriteWithMultipleDests/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+WriteWithMultipleDests/BigQueryBatchFileLoads/FlattenPartitions+WriteWithMultipleDests/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)+WriteWithMultipleDests/BigQueryBatchFileLoads/MapTuple(<lambda at bigquery_file_loads.py:1135>)+WriteWithMultipleDests/BigQueryBatchFileLoads/ParDo(UpdateDestinationSchema)+WriteWithMultipleDests/BigQueryBatchFileLoads/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-22T23:32:20.208Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201 2024-01-22T23:34:23.483Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:151 Job 2024-01-22_15_28_59-17156436973935848627 is in state JOB_STATE_FAILED
[31m[1mERROR [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:777 Console URL: https://console.cloud.google.com/dataflow/jobs/<RegionId>/2024-01-22_15_28_59-17156436973935848627?project=<ProjectId>
[32mINFO [0m apache_beam.io.gcp.bigquery_file_loads_test:bigquery_file_loads_test.py:1242 Deleting dataset python_bq_file_loads_1705966135d935b4 in project apache-beam-testing
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery.py:2603
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2603: DeprecationWarning: invalid escape sequence \#
"""Read data from BigQuery.
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_batch_copy_jobs_with_no_input_schema[0m - AssertionError:
Expected: (<[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]>)
but: <[<BigqueryFullResultMatcher(Expected data is [(0,), (1,), (2,), (3,), (4,)])>, <BigqueryFullResultMatcher(Expected data is [(5,), (6,), (7,), (8,), (9,)])>]> was <DataflowPipelineResult 2024-01-22_15_22_16-4883707003561850353 DONE>
[31mFAILED[0m apache_beam/io/gcp/bigquery_file_loads_test.py::[1mBigQueryFileLoadsIT::test_multiple_destinations_transform[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 23:32:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 1351, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 1354, in apache_beam.runners.worker.operations.FlattenOperation.process
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 23:32:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteWithMultipleDestsFreely/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-71']
[31m====== [31m[1m2 failed[0m, [32m89 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m[31m in 5902.66s (1:38:22)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 44m 45s
222 actionable tasks: 160 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/4nuoqfulyoffg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python39 #2797
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2797/display/redirect>
Changes:
------------------------------------------
[...truncated 8.73 MB...]
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 02:40:16 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 02:40:16 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'WriteTeamScoreSums/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-74']
[31mFAILED[0m apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::[1mBigqueryTornadoesIT::test_bigquery_tornadoes_it[0m - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 02:47:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 311, in _execute
response = task()
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 386, in <lambda>
lambda: self.create_worker().do_instruction(request), request)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 650, in do_instruction
return getattr(self, request_type)(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/sdk_worker.py", line 688, in process_bundle
bundle_processor.process_bundle(instruction_id))
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1113, in process_bundle
input_op_by_transform_id[element.transform_id].process_encoded(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/runners/worker/bundle_processor.py", line 237, in process_encoded
self.output(decoded_value)
File "apache_beam/runners/worker/operations.py", line 570, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 572, in apache_beam.runners.worker.operations.Operation.output
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1526, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 636, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 1621, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "apache_beam/runners/common.py", line 1736, in apache_beam.runners.common._OutputHandler._write_value_to_tag
File "apache_beam/runners/worker/operations.py", line 263, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 266, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
File "apache_beam/runners/worker/operations.py", line 953, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/worker/operations.py", line 954, in apache_beam.runners.worker.operations.DoOperation.process
File "apache_beam/runners/common.py", line 1437, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 1547, in apache_beam.runners.common.DoFnRunner._reraise_augmented
File "apache_beam/runners/common.py", line 1435, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 851, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 995, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "apache_beam/runners/common.py", line 1611, in apache_beam.runners.common._OutputHandler.handle_process_outputs
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 750, in process
job_reference = self.bq_wrapper.perform_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 1022, in perform_load_job
return self._insert_load_job(
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/utils/retry.py", line 298, in wrapper
return fun(*args, **kwargs)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 559, in _insert_load_job
return self._start_job(request, stream=source_stream).jobReference
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 578, in _start_job
response = self.client.jobs.Insert(request, upload=upload)
File "/opt/apache/beam-venv/beam-venv-worker-sdk-0-0/lib/python3.9/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 356, in Insert
return self._RunMethod(
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.9/site-packages/apitools/base/py/base_api.py", line 603, in __ProcessHttpResponse
raise exceptions.HttpError.FromResponse(
RuntimeError: apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 22 Jan 2024 02:47:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '318', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Load configuration must specify at least one source URI",
"errors": [
{
"message": "Load configuration must specify at least one source URI",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}
> [while running 'Write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)-ptransform-82']
[31mERROR[0m apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::[1mBigqueryTornadoesIT::test_bigquery_tornadoes_it[0m - apache_beam.io.gcp.tests.utils.GcpTestIOError: BigQuery table does not exist: apache-beam-testing.BigQueryTornadoesIT.monthly_tornadoes_1705891403239
[31m== [31m[1m4 failed[0m, [32m87 passed[0m, [33m51 skipped[0m, [33m9 warnings[0m, [31m[1m1 error[0m[31m in 5814.61s (1:36:54)[0m[31m ==[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 43m 27s
222 actionable tasks: 156 executed, 62 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/2edkaz2wokr44
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org