You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/09/28 16:07:33 UTC
Build failed in Jenkins:
beam_PerformanceTests_BiqQueryIO_Read_Python #120
See <https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/120/display/redirect?page=changes>
Changes:
[nosacky] Fix up-to-date checking mechanism on Tox Task.
[noreply] [BEAM-10481] Ensure registration of the accumulator occurs. (#12850)
------------------------------------------
[...truncated 62.43 KB...]
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 28 Sep 2020 16:06:20 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:23.164Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:24.390Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 28 Sep 2020 16:06:22 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:26.592Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 28 Sep 2020 16:06:24 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:27.725Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 28 Sep 2020 16:06:26 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.031Z: JOB_MESSAGE_BASIC: Finished operation Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.094Z: JOB_MESSAGE_DEBUG: Executing failure step failure41
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.118Z: JOB_MESSAGE_ERROR: Workflow failed. Causes: S02:Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write failed., Internal Issue (594d4ecc04c25b66): 63963027:24514
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.210Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.271Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:06:28.298Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:07:19.043Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 5 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:07:19.076Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-28T16:07:19.112Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2020-09-28_09_00_54-9099483125819370274 is in state JOB_STATE_FAILED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_perf_test.py",> line 139, in <module>
BigQueryReadPerfTest().run()
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Mon, 28 Sep 2020 16:06:26 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 12s
5 actionable tasks: 5 executed
Publishing build scan...
https://gradle.com/s/bb5o5n35w6il4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_BiqQueryIO_Read_Python #123
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/123/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_PerformanceTests_BiqQueryIO_Read_Python #122
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/122/display/redirect?page=changes>
Changes:
[tysonjh] Fix Java nightly snapshot build.
[sychen] Fix failed GroupIntoBatchesOverrideTest
[Kyle Weaver] [BEAM-10971] Redirect contribute/design-documents to cwiki.
[Kyle Weaver] [BEAM-10671] Add environment configuration fields as a repeated pipeline
[noreply] [BEAM-8024] Add JPMS E2E test (#12899)
[srohde] Adds duration string to the ib.show and ib.collect
[Kyle Weaver] [BEAM-10953] Add errorprone-slf4j plugin.
[Kyle Weaver] [BEAM-10953] Fix errorprone-slf4j errors.
[noreply] [BEAM-8106] Add version to java container image name (#12505)
[Robert Bradshaw] Remove experimental declarations from fileio.
[noreply] [BEAM-10677] Fix @SchemaFieldName in AutoValueSchema (#12520)
------------------------------------------
[...truncated 63.87 KB...]
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 30 Sep 2020 16:07:06 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:14.539Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 30 Sep 2020 16:07:12 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:15.443Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:17.460Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 30 Sep 2020 16:07:15 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:18.622Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 30 Sep 2020 16:07:17 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.194Z: JOB_MESSAGE_BASIC: Finished operation Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.339Z: JOB_MESSAGE_DEBUG: Executing failure step failure41
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.382Z: JOB_MESSAGE_ERROR: Workflow failed. Causes: S02:Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write failed., Internal Issue (d2f0afcda6d0dc90): 63963027:24514
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.471Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.609Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:07:19.641Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:08:13.787Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 5 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:08:14.070Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-30T16:08:14.263Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2020-09-30_09_00_55-10579726144912028366 is in state JOB_STATE_FAILED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_perf_test.py",> line 139, in <module>
BigQueryReadPerfTest().run()
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Wed, 30 Sep 2020 16:07:17 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 3s
5 actionable tasks: 5 executed
Publishing build scan...
https://gradle.com/s/nirsfj7u3aid6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins:
beam_PerformanceTests_BiqQueryIO_Read_Python #121
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/121/display/redirect?page=changes>
Changes:
[Robert Bradshaw] [BEAM-10978] Fix bug with map type inference.
[noreply] Update test_get_python_sdk_name to supported python version (#12950)
[noreply] [BEAM-10882] Update Snowflake docs (#12823)
------------------------------------------
[...truncated 62.94 KB...]
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 29 Sep 2020 16:06:24 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:27.317Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:28.221Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 29 Sep 2020 16:06:26 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:30.394Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 29 Sep 2020 16:06:28 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:32.632Z: JOB_MESSAGE_ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 29 Sep 2020 16:06:30 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:32.930Z: JOB_MESSAGE_BASIC: Finished operation Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:33.003Z: JOB_MESSAGE_DEBUG: Executing failure step failure41
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:33.036Z: JOB_MESSAGE_ERROR: Workflow failed. Causes: S02:Read from BigQuery/ReadFromBigQuery/Read+Read from BigQuery/ReadFromBigQuery/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+Count messages+Measure time+Count/CombineGlobally(CountCombineFn)/KeyWithVoid+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey+Count/CombineGlobally(CountCombineFn)/CombinePerKey/Combine/Partial+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Reify+Count/CombineGlobally(CountCombineFn)/CombinePerKey/GroupByKey/Write failed., Internal Issue (8b6e4fa439ffa6dd): 63963027:24514
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:33.109Z: JOB_MESSAGE_DETAILED: Cleaning up.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:33.185Z: JOB_MESSAGE_DEBUG: Starting **** pool teardown.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:06:33.212Z: JOB_MESSAGE_BASIC: Stopping **** pool...
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:07:32.042Z: JOB_MESSAGE_DETAILED: Autoscaling: Resized **** pool from 5 to 0.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:07:32.078Z: JOB_MESSAGE_BASIC: Worker pool stopped.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-09-29T16:07:32.105Z: JOB_MESSAGE_DEBUG: Tearing down pending resources...
INFO:apache_beam.runners.dataflow.dataflow_runner:Job 2020-09-29_09_00_45-2229782615038240584 is in state JOB_STATE_FAILED
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_perf_test.py",> line 139, in <module>
BigQueryReadPerfTest().run()
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/load_test.py",> line 152, in run
self.result.wait_until_finish(duration=self.timeout_ms)
File "<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1650, in wait_until_finish
self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_****/batch****.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 226, in execute
self._split_task)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 234, in _perform_source_split_considering_api_limits
desired_bundle_size)
File "/usr/local/lib/python3.7/site-packages/dataflow_****/executor.py", line 271, in _perform_source_split
for split in source.split(desired_bundle_size):
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 807, in split
schema, metadata_list = self._export_files(bq)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py", line 879, in _export_files
include_header=False)
File "/usr/local/lib/python3.7/site-packages/apache_beam/utils/retry.py", line 242, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 797, in perform_extract_job
response = self.client.jobs.Insert(request)
File "/usr/local/lib/python3.7/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 346, in Insert
upload=upload, upload_config=upload_config)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
return self.ProcessHttpResponse(method_config, http_response, request)
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
self.__ProcessHttpResponse(method_config, http_response, request))
File "/usr/local/lib/python3.7/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
http_response, method_config=method_config, request=request)
apitools.base.py.exceptions.HttpBadRequestError: HttpError accessing <https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'date': 'Tue, 29 Sep 2020 16:06:30 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'status': '400', 'content-length': '267', '-content-encoding': 'gzip'}>, content <{
"error": {
"code": 400,
"message": "Required parameter is missing",
"errors": [
{
"message": "Required parameter is missing",
"domain": "global",
"reason": "required"
}
],
"status": "INVALID_ARGUMENT"
}
}
>
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '<https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'> line: 58
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 26s
5 actionable tasks: 5 executed
Publishing build scan...
https://gradle.com/s/c6btykkulxw26
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org