You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/11/25 01:57:59 UTC
Build failed in Jenkins: beam_PostCommit_Python36 #4643
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4643/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify DataStreamsDecoder by moving hasNext logic to next
------------------------------------------
[...truncated 51.33 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 173 warnings in 6657.33 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 28s
211 actionable tasks: 168 executed, 39 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pe6yujpl632g6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4702
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4702/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174)
[noreply] [BEAM-12561] method truncate on series and dataframe (#15833)
------------------------------------------
[...truncated 31.69 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 178 warnings in 6489.02 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 17s
211 actionable tasks: 161 executed, 46 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5k4i72zwun63s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4701
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4701/display/redirect>
Changes:
------------------------------------------
[...truncated 28.09 MB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 519823551
}
message: "Renamed 1 shards in 0.16 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 534731149
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 534929752
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 535010337
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 535081624
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 536590099
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639052023
nanos: 536757230
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.703077554702759 seconds.
INFO:root:Successfully completed job in 10.703077554702759 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39377
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc57660c620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fc57660c6a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc57660cd90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempmcz5oyuz/artifactspon7qh3t' '--job-port' '53021' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43337'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41705'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53021'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_18fb1dd0-3606-4d29-a6a3-ba7edf0cf12c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_18fb1dd0-3606-4d29-a6a3-ba7edf0cf12c.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_18fb1dd0-3606-4d29-a6a3-ba7edf0cf12c.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_18fb1dd0-3606-4d29-a6a3-ba7edf0cf12c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:51 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1209121351-af84a887_dac6791c-9f2a-40be-be64-e86005c82e88'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:51 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1209121351-af84a887_dac6791c-9f2a-40be-be64-e86005c82e88'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:51 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:52 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:53 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:53 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1209121351-af84a887_dac6791c-9f2a-40be-be64-e86005c82e88 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45045.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42253.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34449
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209121351-af84a887_dac6791c-9f2a-40be-be64-e86005c82e88: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 12:13:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209121351-af84a887_dac6791c-9f2a-40be-be64-e86005c82e88 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639052035.816991315","description":"Error received from peer ipv4:127.0.0.1:34449","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1639052035.817035176","description":"Error received from peer ipv4:127.0.0.1:45045","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639052035.816991315","description":"Error received from peer ipv4:127.0.0.1:34449","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639052035.817008716","description":"Error received from peer ipv4:127.0.0.1:42253","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 12s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/knpxpsm7qzqgg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4700
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4700/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13015] Start integrating a process wide cache. (#16130)
------------------------------------------
[...truncated 1.65 MB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 90464830
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 198081731
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 209130525
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 209375143
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 209506511
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 209593772
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 211367368
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639030796
nanos: 211495876
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 15.836891651153564 seconds.
INFO:root:Successfully completed job in 15.836891651153564 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35189
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f057c2e7620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f057c2e76a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f057c2e7d90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempdzeqzle8/artifacts9rgn02dr' '--job-port' '35743' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44915'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44575'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:35743'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_28067749-082d-40ff-8d17-b63c3f1573ee.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_28067749-082d-40ff-8d17-b63c3f1573ee.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_28067749-082d-40ff-8d17-b63c3f1573ee.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_28067749-082d-40ff-8d17-b63c3f1573ee.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:03 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1209062003-c722bcb_2b5dc864-4577-4966-93da-7d83e2e929bb'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:03 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1209062003-c722bcb_2b5dc864-4577-4966-93da-7d83e2e929bb'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/12/09 06:20:04 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1209062003-c722bcb_2b5dc864-4577-4966-93da-7d83e2e929bb on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36553.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35245.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43813
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209062003-c722bcb_2b5dc864-4577-4966-93da-7d83e2e929bb: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 06:20:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209062003-c722bcb_2b5dc864-4577-4966-93da-7d83e2e929bb finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639030807.579253360","description":"Error received from peer ipv4:127.0.0.1:43813","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1639030807.579268664","description":"Error received from peer ipv4:127.0.0.1:36553","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639030807.579253360","description":"Error received from peer ipv4:127.0.0.1:43813","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639030807.579269671","description":"Error received from peer ipv4:127.0.0.1:35245","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 2s
211 actionable tasks: 156 executed, 51 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/yejil33mvmt4s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4699
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4699/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171)
[noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK
[noreply] [BEAM-11936] Fix errorprone warnings (#15890)
------------------------------------------
[...truncated 31.88 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 178 warnings in 6355.49 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 9m 22s
211 actionable tasks: 182 executed, 25 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/tjdajdu6hwu6q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4698
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4698/display/redirect?page=changes>
Changes:
[stranniknm] fix playground frontend licences
[noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change
[noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines
[noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness..
------------------------------------------
[...truncated 736.93 KB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 180 warnings in 6443.66 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:io:mongodb:compileJava'.
> Compilation failed; see the compiler error output for details.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 13m 59s
201 actionable tasks: 172 executed, 25 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nsbdwjpy6f4hg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4697
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4697/display/redirect?page=changes>
Changes:
[heejong] [BEAM-13092] Adding dummy external transform translators for Dataflow
[melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content
------------------------------------------
[...truncated 26.53 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 524912834
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 644044876
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 669468879
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 669658899
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 669744968
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 669821023
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 670296669
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638965776
nanos: 670401096
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.855908393859863 seconds.
INFO:root:Successfully completed job in 6.855908393859863 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44087
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2a3fe3a620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2a3fe3a6a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2a3fe3ad90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp3lqer28u/artifactspnmk_u1w' '--job-port' '35883' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45579'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36113'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:35883'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_8964dc86-f00a-463b-96a8-ec944322156c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_8964dc86-f00a-463b-96a8-ec944322156c.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_8964dc86-f00a-463b-96a8-ec944322156c.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_8964dc86-f00a-463b-96a8-ec944322156c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1208121622-8f7dcb61_96cba368-2bfc-43f3-b8cd-ac3bc7b44488'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:23 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1208121622-8f7dcb61_96cba368-2bfc-43f3-b8cd-ac3bc7b44488'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:23 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:23 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:24 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:24 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1208121622-8f7dcb61_96cba368-2bfc-43f3-b8cd-ac3bc7b44488 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39723.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39479.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37323
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208121622-8f7dcb61_96cba368-2bfc-43f3-b8cd-ac3bc7b44488: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 12:16:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208121622-8f7dcb61_96cba368-2bfc-43f3-b8cd-ac3bc7b44488 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638965786.807968335","description":"Error received from peer ipv4:127.0.0.1:37323","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638965786.807968335","description":"Error received from peer ipv4:127.0.0.1:37323","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638965786.807970018","description":"Error received from peer ipv4:127.0.0.1:39479","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638965786.808016129","description":"Error received from peer ipv4:127.0.0.1:39723","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 35s
211 actionable tasks: 161 executed, 46 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/eihj62iyyzjvo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4696
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4696/display/redirect?page=changes>
Changes:
[relax] don't store unserializable DatasetService as a member variable
[dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577
[noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164)
[noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support
------------------------------------------
[...truncated 751.51 KB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 671622753
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 684498786
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 684811115
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 684979200
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 685087442
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 685634851
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944889
nanos: 685765743
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.101297855377197 seconds.
INFO:root:Successfully completed job in 7.101297855377197 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42727
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb5c8845620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb5c88456a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb5c8845d90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempaxrgtteg/artifacts531atfzm' '--job-port' '46933' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35549'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40865'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:46933'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d9dc6a61-d771-4a45-9039-733c155d65a6.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d9dc6a61-d771-4a45-9039-733c155d65a6.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d9dc6a61-d771-4a45-9039-733c155d65a6.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d9dc6a61-d771-4a45-9039-733c155d65a6.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1208062816-6b2946fa_9a7c0254-448e-4a38-9b76-01d1a7d4cfd9'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:16 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1208062816-6b2946fa_9a7c0254-448e-4a38-9b76-01d1a7d4cfd9'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:17 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:17 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:18 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:18 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1208062816-6b2946fa_9a7c0254-448e-4a38-9b76-01d1a7d4cfd9 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39117.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45059.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37817
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208062816-6b2946fa_9a7c0254-448e-4a38-9b76-01d1a7d4cfd9: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:28:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208062816-6b2946fa_9a7c0254-448e-4a38-9b76-01d1a7d4cfd9 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944901.995557600","description":"Error received from peer ipv4:127.0.0.1:37817","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944901.995557600","description":"Error received from peer ipv4:127.0.0.1:37817","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944901.995672438","description":"Error received from peer ipv4:127.0.0.1:39117","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944901.995591821","description":"Error received from peer ipv4:127.0.0.1:45059","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 2m 49s
211 actionable tasks: 167 executed, 40 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jaiuipl5jjnjg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4695
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4695/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-13388] Fix broken google cloud dlp test.
[zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of
[msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test.
[noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up
[noreply] [BEAM-12976] Implement pipeline visitor to get global field access in…
------------------------------------------
[...truncated 164.52 KB...]
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:16:58 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:16:59 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:00 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:00 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1208001658-3d660e62_a16df8e4-8cc8-483c-8825-4b47d2b25ae3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37631.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40111.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38381
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208001658-3d660e62_a16df8e4-8cc8-483c-8825-4b47d2b25ae3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 00:17:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208001658-3d660e62_a16df8e4-8cc8-483c-8825-4b47d2b25ae3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638922624.158391798","description":"Error received from peer ipv4:127.0.0.1:38381","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638922624.158391798","description":"Error received from peer ipv4:127.0.0.1:38381","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638922624.161751626","description":"Error received from peer ipv4:127.0.0.1:37631","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638922624.161748452","description":"Error received from peer ipv4:127.0.0.1:40111","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638922686.020838080","description":"Error received from peer ipv4:127.0.0.1:44669","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638922730.735289193","description":"Error received from peer ipv4:127.0.0.1:33249","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638922735.706036416","description":"Error received from peer ipv4:127.0.0.1:33999","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638922786.767272514","description":"Error received from peer ipv4:127.0.0.1:34095","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638922913.002945404","description":"Error received from peer ipv4:127.0.0.1:46637","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 3m 39s
211 actionable tasks: 175 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6jijkv7d6dzns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4694
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4694/display/redirect?page=changes>
Changes:
[stranniknm] [BEAM-13112]: playground embedded version
[noreply] [BEAM-13236] Properly close kinesis producer on teardown (#15955)
[noreply] Merge pull request #16150 from [BEAM-13396][Playground][Bugfix] Issues
[noreply] Merge pull request #16148 from [BEAM-13394][Playground] [Bugfix] Fix
[noreply] Avoid overriding explicit portable job submission disabling. (#16143)
[noreply] Merge pull request #16151 from [BEAM-13350][Playground] Support running
------------------------------------------
[...truncated 29.55 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 679118871
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 798793315
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 823611974
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 823875427
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 823973417
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 824050188
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 824549198
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638901091
nanos: 824666976
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.390739917755127 seconds.
INFO:root:Successfully completed job in 6.390739917755127 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34639
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6eb0b3c620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f6eb0b3c6a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6eb0b3cd90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzpw1nj0f/artifactsdce6_c0g' '--job-port' '50415' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:50415.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33013'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42903'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50415'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:50415.
WARNING:root:Waiting for grpc channel to be ready at localhost:50415.
WARNING:root:Waiting for grpc channel to be ready at localhost:50415.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_42066563-a9d7-4eb4-95fe-9d53536ac5ed.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_42066563-a9d7-4eb4-95fe-9d53536ac5ed.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_42066563-a9d7-4eb4-95fe-9d53536ac5ed.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_42066563-a9d7-4eb4-95fe-9d53536ac5ed.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:25 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1207181825-9d372e17_ea382e50-8cb6-4687-8f31-c9010cc6f478'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:25 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1207181825-9d372e17_ea382e50-8cb6-4687-8f31-c9010cc6f478'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1207181825-9d372e17_ea382e50-8cb6-4687-8f31-c9010cc6f478 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40391.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38171.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38861
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207181825-9d372e17_ea382e50-8cb6-4687-8f31-c9010cc6f478: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 18:18:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207181825-9d372e17_ea382e50-8cb6-4687-8f31-c9010cc6f478 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638901114.887599621","description":"Error received from peer ipv4:127.0.0.1:40391","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638901114.887570898","description":"Error received from peer ipv4:127.0.0.1:38171","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638901114.887570117","description":"Error received from peer ipv4:127.0.0.1:38861","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638901114.887570117","description":"Error received from peer ipv4:127.0.0.1:38861","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 51s
211 actionable tasks: 150 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2xm2r56jxkale
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4693
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4693/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify fake state API testing client and make it
------------------------------------------
[...truncated 23.05 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 239347219
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 343649387
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358120918
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358300685
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358387231
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358455896
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358828783
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638879197
nanos: 358922481
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 9.294158935546875 seconds.
INFO:root:Successfully completed job in 9.294158935546875 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43849
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb296011620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb2960116a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb296011d90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp78gozapu/artifactsilf3sr6e' '--job-port' '39523' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38081'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44717'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39523'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_7e34cd53-fe75-4f9f-9ecb-49e36d6b3887.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_7e34cd53-fe75-4f9f-9ecb-49e36d6b3887.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_7e34cd53-fe75-4f9f-9ecb-49e36d6b3887.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_7e34cd53-fe75-4f9f-9ecb-49e36d6b3887.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1207121322-fcfaf1c9_da625447-44bc-43d8-9d06-63cc20fbc88f'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:22 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1207121322-fcfaf1c9_da625447-44bc-43d8-9d06-63cc20fbc88f'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:23 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:23 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:24 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:24 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1207121322-fcfaf1c9_da625447-44bc-43d8-9d06-63cc20fbc88f on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42089.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34449.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39323
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207121322-fcfaf1c9_da625447-44bc-43d8-9d06-63cc20fbc88f: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 12:13:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207121322-fcfaf1c9_da625447-44bc-43d8-9d06-63cc20fbc88f finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638879206.746466026","description":"Error received from peer ipv4:127.0.0.1:39323","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638879206.746503018","description":"Error received from peer ipv4:127.0.0.1:42089","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638879206.746466026","description":"Error received from peer ipv4:127.0.0.1:39323","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638879206.746482861","description":"Error received from peer ipv4:127.0.0.1:34449","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 46m 28s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/hhoypwsjeajts
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4692
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4692/display/redirect?page=changes>
Changes:
[Daniel Oliveira] Minor Gradle Fixups: Removing unneeded dependency in expansion service.
[melissapa] [BEAM-11758] Update basics page: Window, Watermark
------------------------------------------
[...truncated 1.61 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 527542829
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 634610891
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 646443367
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 646684646
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 646792173
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 646881580
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 647488355
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638858609
nanos: 647638559
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 29.025113105773926 seconds.
INFO:root:Successfully completed job in 29.025113105773926 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41625
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3ee6a79620> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3ee6a796a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3ee6a79d90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp41yx0o69/artifactsaegh9c17' '--job-port' '34341' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:34341.
WARNING:root:Waiting for grpc channel to be ready at localhost:34341.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36985'
WARNING:root:Waiting for grpc channel to be ready at localhost:34341.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43061'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34341'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:34341.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_b4f7dd14-64fe-4546-b825-d364a7326448.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_b4f7dd14-64fe-4546-b825-d364a7326448.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_b4f7dd14-64fe-4546-b825-d364a7326448.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_b4f7dd14-64fe-4546-b825-d364a7326448.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:24 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1207063024-57693ee4_b1683508-ff28-4c90-9a8c-fb19c5024991'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:25 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1207063024-57693ee4_b1683508-ff28-4c90-9a8c-fb19c5024991'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:27 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1207063024-57693ee4_b1683508-ff28-4c90-9a8c-fb19c5024991 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45307.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34927.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38467
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207063024-57693ee4_b1683508-ff28-4c90-9a8c-fb19c5024991: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 06:30:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207063024-57693ee4_b1683508-ff28-4c90-9a8c-fb19c5024991 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638858635.771545663","description":"Error received from peer ipv4:127.0.0.1:38467","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638858635.771545663","description":"Error received from peer ipv4:127.0.0.1:38467","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638858635.771594022","description":"Error received from peer ipv4:127.0.0.1:45307","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638858635.771573004","description":"Error received from peer ipv4:127.0.0.1:34927","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 12s
211 actionable tasks: 148 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bmx5sgk57crxw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4691
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4691/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-13380] Fix broken seedjob.
------------------------------------------
[...truncated 28.35 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638836085
nanos: 908075332
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 13186454
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27009725
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27210235
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27286529
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27359008
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27860403
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638836086
nanos: 27973651
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.6302080154418945 seconds.
INFO:root:Successfully completed job in 6.6302080154418945 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43863
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4bc121e6a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f4bc121e730> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4bc121ee18> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempfnhz5p2n/artifactsqq0h54se' '--job-port' '40211' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36375'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42139'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40211'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9e5812d2-ab98-498d-bffe-8f5a5b2dfd21.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9e5812d2-ab98-498d-bffe-8f5a5b2dfd21.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9e5812d2-ab98-498d-bffe-8f5a5b2dfd21.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9e5812d2-ab98-498d-bffe-8f5a5b2dfd21.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1207001451-a39d0f87_c69f5776-d44c-479d-b649-123c1dca3a51'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:51 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1207001451-a39d0f87_c69f5776-d44c-479d-b649-123c1dca3a51'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:52 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:52 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:53 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:53 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1207001451-a39d0f87_c69f5776-d44c-479d-b649-123c1dca3a51 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35837.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43563.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35431
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207001451-a39d0f87_c69f5776-d44c-479d-b649-123c1dca3a51: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/07 00:14:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1207001451-a39d0f87_c69f5776-d44c-479d-b649-123c1dca3a51 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638836096.111172725","description":"Error received from peer ipv4:127.0.0.1:35431","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638836096.111172725","description":"Error received from peer ipv4:127.0.0.1:35431","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638836096.111183328","description":"Error received from peer ipv4:127.0.0.1:43563","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638836096.111197364","description":"Error received from peer ipv4:127.0.0.1:35837","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 17s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/di3ulb7zelhfu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4690
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4690/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13363][Playground]
[aydar.zaynutdinov] [BEAM-13357][Playground]
[aydar.zaynutdinov] [BEAM-13357][Playground]
[aydar.zaynutdinov] [BEAM-13357][Playground]
[noreply] Merge pull request #16137 from [BEAM-13381][Playground][Bugfix] Update
[noreply] Merge pull request #16107 from [BEAM-13332] [Playground] Add logs output
[noreply] Merge pull request #16109 from [BEAM-13330][Playground] Save Java logs
[noreply] Merge pull request #16125 from [BEAM-13322][Playground] Support Java
------------------------------------------
[...truncated 31.82 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 190 warnings in 6428.28 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 16s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/oa4ydedkka4o2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4689
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4689/display/redirect>
Changes:
------------------------------------------
[...truncated 61.66 MB...]
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 186 warnings in 6299.69 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 8s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xmmpb3hxkhkaw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4688
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4688/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16118 from [BEAM-13362][Playground] python ci cd
------------------------------------------
[...truncated 30.34 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 4 failed, 54 passed, 11 skipped, 190 warnings in 6362.49 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 31s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/b4nxet55xn3ly
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4687
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4687/display/redirect>
Changes:
------------------------------------------
[...truncated 29.36 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 540528535
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 692225456
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 716268301
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 716506481
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 716591596
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 716670513
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 717319011
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638749689
nanos: 717462301
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.695610523223877 seconds.
INFO:root:Successfully completed job in 7.695610523223877 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44403
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4987e656a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f4987e65730> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4987e65e18> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempkn73h6n1/artifacts0qxbi0kf' '--job-port' '40671' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:40671.
WARNING:root:Waiting for grpc channel to be ready at localhost:40671.
WARNING:root:Waiting for grpc channel to be ready at localhost:40671.
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46045'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36829'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40671'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:40671.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_93b2bbf5-1c32-4d5a-a850-8a71d13b7d4c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_93b2bbf5-1c32-4d5a-a850-8a71d13b7d4c.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_93b2bbf5-1c32-4d5a-a850-8a71d13b7d4c.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:05 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_93b2bbf5-1c32-4d5a-a850-8a71d13b7d4c.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:06 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1206001505-121d944f_f152151c-5cbb-4edc-b133-376d85854636'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:06 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1206001505-121d944f_f152151c-5cbb-4edc-b133-376d85854636'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:07 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:09 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:13 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:13 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1206001505-121d944f_f152151c-5cbb-4edc-b133-376d85854636 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45289.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37239.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37281
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1206001505-121d944f_f152151c-5cbb-4edc-b133-376d85854636: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/06 00:15:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1206001505-121d944f_f152151c-5cbb-4edc-b133-376d85854636 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638749717.750323619","description":"Error received from peer ipv4:127.0.0.1:37281","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638749717.750323619","description":"Error received from peer ipv4:127.0.0.1:37281","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638749717.750360843","description":"Error received from peer ipv4:127.0.0.1:45289","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638749717.750323169","description":"Error received from peer ipv4:127.0.0.1:37239","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 36s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jtwpawbvwpv2g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4686
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4686/display/redirect>
Changes:
------------------------------------------
[...truncated 1.86 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 121768951
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 227486371
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 240911722
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 241077184
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 241155147
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 241236686
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 241698980
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638728400
nanos: 241806030
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.026233434677124 seconds.
INFO:root:Successfully completed job in 7.026233434677124 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35135
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fcbebc2c6a8> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fcbebc2c730> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fcbebc2ce18> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp6y2n4znk/artifactsz1o4_40v' '--job-port' '39437' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34209'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38855'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39437'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:39437.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c76d9333-0784-4629-a1e0-694c12f4ea24.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c76d9333-0784-4629-a1e0-694c12f4ea24.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_c76d9333-0784-4629-a1e0-694c12f4ea24.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c76d9333-0784-4629-a1e0-694c12f4ea24.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1205182008-3d29f7fa_2012ceee-0e3d-459d-901a-2da196cfe12b'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1205182008-3d29f7fa_2012ceee-0e3d-459d-901a-2da196cfe12b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:09 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:09 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/12/05 18:20:10 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:10 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:10 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1205182008-3d29f7fa_2012ceee-0e3d-459d-901a-2da196cfe12b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33201.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42515.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45745
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1205182008-3d29f7fa_2012ceee-0e3d-459d-901a-2da196cfe12b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/05 18:20:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1205182008-3d29f7fa_2012ceee-0e3d-459d-901a-2da196cfe12b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638728413.666327314","description":"Error received from peer ipv4:127.0.0.1:42515","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638728413.666340309","description":"Error received from peer ipv4:127.0.0.1:33201","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638728413.666295200","description":"Error received from peer ipv4:127.0.0.1:45745","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638728413.666295200","description":"Error received from peer ipv4:127.0.0.1:45745","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 12s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/gddpm2zmr2qkk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4685
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4685/display/redirect>
Changes:
------------------------------------------
[...truncated 29.31 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 845211744
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 966223001
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 989495515
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 989718675
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 989829778
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 989912509
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 990481853
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638706500
nanos: 990619182
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.714709520339966 seconds.
INFO:root:Successfully completed job in 7.714709520339966 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34359
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f6f21c5a400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f6f21c5a488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f6f21c5ab70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-templcgmb1is/artifactsmgqkpxlb' '--job-port' '44679' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:06 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35007'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:06 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43111'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:06 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44679'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:06 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_3648bb79-c4dd-4d4b-9095-c91d9b8aa04f.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_3648bb79-c4dd-4d4b-9095-c91d9b8aa04f.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_3648bb79-c4dd-4d4b-9095-c91d9b8aa04f.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:07 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_3648bb79-c4dd-4d4b-9095-c91d9b8aa04f.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1205121508-c6587c87_a5f5c626-64a7-45bd-85d3-c82909e60e70'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:08 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1205121508-c6587c87_a5f5c626-64a7-45bd-85d3-c82909e60e70'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:08 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:09 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:09 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1205121508-c6587c87_a5f5c626-64a7-45bd-85d3-c82909e60e70 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37183.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35763.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:32803
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1205121508-c6587c87_a5f5c626-64a7-45bd-85d3-c82909e60e70: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/05 12:15:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1205121508-c6587c87_a5f5c626-64a7-45bd-85d3-c82909e60e70 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638706512.885632985","description":"Error received from peer ipv4:127.0.0.1:32803","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638706512.885632985","description":"Error received from peer ipv4:127.0.0.1:32803","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638706512.885642827","description":"Error received from peer ipv4:127.0.0.1:35763","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638706512.885730890","description":"Error received from peer ipv4:127.0.0.1:37183","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 47s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mflarkyrp256y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4684
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4684/display/redirect>
Changes:
------------------------------------------
[...truncated 35.36 MB...]
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 180 warnings in 6275.29 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 48m 33s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mr6ho73awjxli
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4683
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4683/display/redirect>
Changes:
------------------------------------------
[...truncated 30.25 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 188 warnings in 6363.24 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 59s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2mmt3e6l2g5d6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4682
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4682/display/redirect>
Changes:
------------------------------------------
[...truncated 24.88 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638641723
nanos: 976049184
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 81363201
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 93569755
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 93852519
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 93955039
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 94048976
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 95814943
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638641724
nanos: 95996618
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.4270899295806885 seconds.
INFO:root:Successfully completed job in 6.4270899295806885 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43245
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f414741c400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f414741c488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f414741cb70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temped0ejjqi/artifactstvvkccd7' '--job-port' '45333' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34929'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44967'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:45333'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_eac18579-7745-4844-8319-a3dd3a35d8dd.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_eac18579-7745-4844-8319-a3dd3a35d8dd.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_eac18579-7745-4844-8319-a3dd3a35d8dd.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_eac18579-7745-4844-8319-a3dd3a35d8dd.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1204181530-b48c11a4_7fbe4797-f4f4-4414-9e42-01a1b6e10435'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:30 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1204181530-b48c11a4_7fbe4797-f4f4-4414-9e42-01a1b6e10435'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:31 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:31 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:32 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:32 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1204181530-b48c11a4_7fbe4797-f4f4-4414-9e42-01a1b6e10435 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43773.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41351.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41349
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1204181530-b48c11a4_7fbe4797-f4f4-4414-9e42-01a1b6e10435: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/04 18:15:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1204181530-b48c11a4_7fbe4797-f4f4-4414-9e42-01a1b6e10435 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638641736.154052075","description":"Error received from peer ipv4:127.0.0.1:41349","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638641736.154052075","description":"Error received from peer ipv4:127.0.0.1:41349","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638641736.154071259","description":"Error received from peer ipv4:127.0.0.1:41351","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638641736.154076379","description":"Error received from peer ipv4:127.0.0.1:43773","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 50m 52s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/riymjgochmeqs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4681
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4681/display/redirect>
Changes:
------------------------------------------
[...truncated 34.96 MB...]
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 186 warnings in 6477.60 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 59m 32s
211 actionable tasks: 166 executed, 41 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kegqu3hpncbky
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4680
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4680/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Revert "Revert "Allow wildcards for java class lookup transform
[Robert Bradshaw] Move stand-alone expansion service jar into its own project.
------------------------------------------
[...truncated 2.35 MB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638599392
nanos: 972911357
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 79161167
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 93325138
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 93563079
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 93676805
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 93772888
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 95236301
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638599393
nanos: 95419168
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 26.835319757461548 seconds.
INFO:root:Successfully completed job in 26.835319757461548 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37761
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb43078b400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb43078b488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb43078bb70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp2x4dz0cj/artifactsaecf25we' '--job-port' '60541' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:60541.
WARNING:root:Waiting for grpc channel to be ready at localhost:60541.
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45625'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44851'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60541'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:60541.
WARNING:root:Waiting for grpc channel to be ready at localhost:60541.
WARNING:root:Waiting for grpc channel to be ready at localhost:60541.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e8707047-a7f3-4e5e-9cd3-7ea2f267fabb.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e8707047-a7f3-4e5e-9cd3-7ea2f267fabb.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e8707047-a7f3-4e5e-9cd3-7ea2f267fabb.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e8707047-a7f3-4e5e-9cd3-7ea2f267fabb.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1204063009-d5f5f3ee_3dd82b1d-4c18-4ed9-888a-9307659f9b00'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1204063009-d5f5f3ee_3dd82b1d-4c18-4ed9-888a-9307659f9b00'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:12 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:15 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:15 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1204063009-d5f5f3ee_3dd82b1d-4c18-4ed9-888a-9307659f9b00 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43435.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38213.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46277
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1204063009-d5f5f3ee_3dd82b1d-4c18-4ed9-888a-9307659f9b00: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.17 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/04 06:30:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1204063009-d5f5f3ee_3dd82b1d-4c18-4ed9-888a-9307659f9b00 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638599421.238683949","description":"Error received from peer ipv4:127.0.0.1:38213","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638599421.238690324","description":"Error received from peer ipv4:127.0.0.1:43435","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638599421.238671052","description":"Error received from peer ipv4:127.0.0.1:46277","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638599421.238671052","description":"Error received from peer ipv4:127.0.0.1:46277","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 40s
211 actionable tasks: 162 executed, 45 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/uqpoh64u2eq7u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4679
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4679/display/redirect?page=changes>
Changes:
[noreply] Bump python containers to beam-master-20211202 (#16129)
[noreply] [BEAM-13354, BEAM-13015, BEAM-12802, BEAM-12588] Support prefetch for
[zyichi] Fix failing RecommendationAICatalogItemIT
[Valentyn Tymofieiev] Clarify instructions on how to get contributor list.
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in IO (#16036)
------------------------------------------
[...truncated 31.77 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 180 warnings in 6268.23 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 10s
211 actionable tasks: 172 executed, 35 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/zuztjnw5ramjm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python36 - Build # 4678 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4678 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4678/ to view the results.
beam_PostCommit_Python36 - Build # 4677 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4677 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4677/ to view the results.
beam_PostCommit_Python36 - Build # 4676 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4676 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4676/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python36 #4675
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4675/display/redirect?page=changes>
Changes:
[noreply] Don't pin a particular version of Tensorflow. (#16102)
[noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI
[noreply] [BEAM-13288] improve logging for no rows present error (#16096)
------------------------------------------
[...truncated 53.02 MB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 300189971
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 319520235
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 319696903
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 319817066
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 319925069
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 320336818
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638490958
nanos: 320423841
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.830131530761719 seconds.
INFO:root:Successfully completed job in 5.830131530761719 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34439
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7eff1f94a048> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7eff1f94a0d0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7eff1f94a7b8> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempa4yxj4kl/artifactsbnwyvots' '--job-port' '50515' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34147'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33623'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50515'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9abbaab4-4192-48ac-81b5-b16026f4917b.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9abbaab4-4192-48ac-81b5-b16026f4917b.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9abbaab4-4192-48ac-81b5-b16026f4917b.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9abbaab4-4192-48ac-81b5-b16026f4917b.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1203002243-c9bc485d_07f2dc8b-ce79-4b72-a358-d2f880106e50'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:43 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1203002243-c9bc485d_07f2dc8b-ce79-4b72-a358-d2f880106e50'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:44 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:45 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:45 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1203002243-c9bc485d_07f2dc8b-ce79-4b72-a358-d2f880106e50 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37631.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40623.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43555
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203002243-c9bc485d_07f2dc8b-ce79-4b72-a358-d2f880106e50: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:22:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203002243-c9bc485d_07f2dc8b-ce79-4b72-a358-d2f880106e50 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638490967.922140437","description":"Error received from peer ipv4:127.0.0.1:43555","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638490967.922140437","description":"Error received from peer ipv4:127.0.0.1:43555","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638490967.922179266","description":"Error received from peer ipv4:127.0.0.1:37631","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638490967.922177700","description":"Error received from peer ipv4:127.0.0.1:40623","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 286
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 15s
211 actionable tasks: 148 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vpt2xstzx3jf2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4674
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4674/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13329][Playground]
[aydar.zaynutdinov] [BEAM-13329][Playground]
[alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings
[ilya.kozyrev] Fix pylint issues and apply yapf with Beam config
[ilya.kozyrev] fix white spaces
------------------------------------------
[...truncated 45.71 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 422237873
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 422449350
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 422535657
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 422639846
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 423948764
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469479
nanos: 424093723
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.157965183258057 seconds.
INFO:root:Successfully completed job in 8.157965183258057 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38407
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe3bba12c80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe3bba12d08> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe3bba13488> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temppfnqpv37/artifactsgbz5z8ng' '--job-port' '35733' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:35733.
WARNING:root:Waiting for grpc channel to be ready at localhost:35733.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38047'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44521'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:35733'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:35733.
WARNING:root:Waiting for grpc channel to be ready at localhost:35733.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_57b03283-a49e-45fa-957b-c1acba0c7e18.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_57b03283-a49e-45fa-957b-c1acba0c7e18.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_57b03283-a49e-45fa-957b-c1acba0c7e18.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:53 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_57b03283-a49e-45fa-957b-c1acba0c7e18.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:54 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1202182453-22faa907_9a587ca8-ce15-4c71-987c-78dd801ec304'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:54 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1202182453-22faa907_9a587ca8-ce15-4c71-987c-78dd801ec304'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:55 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:24:56 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:00 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:00 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1202182453-22faa907_9a587ca8-ce15-4c71-987c-78dd801ec304 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35135.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46537.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39073
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:02 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202182453-22faa907_9a587ca8-ce15-4c71-987c-78dd801ec304: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:25:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202182453-22faa907_9a587ca8-ce15-4c71-987c-78dd801ec304 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638469504.837208784","description":"Error received from peer ipv4:127.0.0.1:35135","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469504.837199854","description":"Error received from peer ipv4:127.0.0.1:46537","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469504.837177686","description":"Error received from peer ipv4:127.0.0.1:39073","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469504.837177686","description":"Error received from peer ipv4:127.0.0.1:39073","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 286
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 59m 31s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7o5uhulx4ki3k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4673
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4673/display/redirect>
Changes:
------------------------------------------
[...truncated 43.22 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 474788427
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 489159107
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 489341020
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 489420175
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 489489316
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 489969015
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638447416
nanos: 490083932
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.53389573097229 seconds.
INFO:root:Successfully completed job in 5.53389573097229 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46759
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f5b9e588c80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f5b9e588d08> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f5b9e589488> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp4gknsyqr/artifactsva1_i8yf' '--job-port' '60899' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44563'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46039'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60899'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d6d445e3-9155-4f85-aa90-7f66de5c71f6.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d6d445e3-9155-4f85-aa90-7f66de5c71f6.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d6d445e3-9155-4f85-aa90-7f66de5c71f6.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d6d445e3-9155-4f85-aa90-7f66de5c71f6.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1202121702-a1be4c3d_a542dca5-a42a-4a8c-bb04-632c99352733'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:02 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1202121702-a1be4c3d_a542dca5-a42a-4a8c-bb04-632c99352733'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1202121702-a1be4c3d_a542dca5-a42a-4a8c-bb04-632c99352733 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33757.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37169.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36345
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202121702-a1be4c3d_a542dca5-a42a-4a8c-bb04-632c99352733: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 12:17:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202121702-a1be4c3d_a542dca5-a42a-4a8c-bb04-632c99352733 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638447426.822997284","description":"Error received from peer ipv4:127.0.0.1:37169","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638447426.823016818","description":"Error received from peer ipv4:127.0.0.1:33757","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638447426.822997244","description":"Error received from peer ipv4:127.0.0.1:36345","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638447426.822997244","description":"Error received from peer ipv4:127.0.0.1:36345","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 7s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/lwtwn43cjtsac
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4672
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4672/display/redirect>
Changes:
------------------------------------------
[...truncated 50.85 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (9/16)#0 (dd5f491c3c1b69384822c8aee893caf5) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (9/16) (dd5f491c3c1b69384822c8aee893caf5) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot b78e449716fd38b51d41cfcc972ebbeb.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16)#0 (216abc5e181706e97007c873be7b5949) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16)#0 (216abc5e181706e97007c873be7b5949) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16)#0 (216abc5e181706e97007c873be7b5949) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16) (216abc5e181706e97007c873be7b5949) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService getProcessBundleDescriptor'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: getProcessBundleDescriptor request with id 1-31'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16)#0 (216abc5e181706e97007c873be7b5949) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (10/16) (216abc5e181706e97007c873be7b5949) switched from INITIALIZING to RUNNING.'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:807 Creating insecure state channel for localhost:41985.
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:814 State channel established.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16)#0 (6addb6c0f54a3313b3b755e054450bbb), deploy into slot with allocation id b78e449716fd38b51d41cfcc972ebbeb.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.runners.worker.data_plane:data_plane.py:750 Creating client data channel for localhost:46545
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 4adf9e058f78b1781ed242bef5a4e9b2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16)#0 (8aa1e441171c1a0558b6575eb484a04f), deploy into slot with allocation id 4adf9e058f78b1781ed242bef5a4e9b2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16)#0 (6addb6c0f54a3313b3b755e054450bbb) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16)#0 (6addb6c0f54a3313b3b755e054450bbb) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16)#0 (6addb6c0f54a3313b3b755e054450bbb) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16) (6addb6c0f54a3313b3b755e054450bbb) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16)#0 (6addb6c0f54a3313b3b755e054450bbb) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (11/16) (6addb6c0f54a3313b3b755e054450bbb) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16)#0 (8aa1e441171c1a0558b6575eb484a04f) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16)#0 (8aa1e441171c1a0558b6575eb484a04f) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16)#0 (8aa1e441171c1a0558b6575eb484a04f) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 8b3459eb9492001ee9212f0cd675f47d.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16) (8aa1e441171c1a0558b6575eb484a04f) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16)#0 (8aa1e441171c1a0558b6575eb484a04f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (12/16) (8aa1e441171c1a0558b6575eb484a04f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16)#0 (c4eb89bd412545e341f33eaa8cafa5ff), deploy into slot with allocation id 8b3459eb9492001ee9212f0cd675f47d.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16)#0 (c4eb89bd412545e341f33eaa8cafa5ff) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot f60b326998cde5297b943c7156d4d506.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16)#0 (c4eb89bd412545e341f33eaa8cafa5ff) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16)#0 (c4eb89bd412545e341f33eaa8cafa5ff) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.runners.fnexecution.data.GrpcDataService data'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Beam Fn Data client connected.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16) (c4eb89bd412545e341f33eaa8cafa5ff) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16)#0 (c4eb89bd412545e341f33eaa8cafa5ff) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16)#0 (e74208da0d7e9eb913eb094cc1054cee), deploy into slot with allocation id f60b326998cde5297b943c7156d4d506.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (13/16) (c4eb89bd412545e341f33eaa8cafa5ff) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16)#0 (e74208da0d7e9eb913eb094cc1054cee) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16)#0 (e74208da0d7e9eb913eb094cc1054cee) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16)#0 (e74208da0d7e9eb913eb094cc1054cee) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16)#0 (e74208da0d7e9eb913eb094cc1054cee) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.metrics.groups.TaskMetricGroup getOrAddOperator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: The operator name MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) exceeded the 80 characters length limit and was truncated.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16) (e74208da0d7e9eb913eb094cc1054cee) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (14/16) (e74208da0d7e9eb913eb094cc1054cee) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot f45f7ab554bc746f9657ec398c2d5130.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16)#0 (8f323e274877d7d2c8ff45633a2685e3), deploy into slot with allocation id f45f7ab554bc746f9657ec398c2d5130.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 19c43fd99972ce72ef7ee618806d3eab.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16)#0 (f7461d8190b6de4a64a11ea453cc24f4), deploy into slot with allocation id 19c43fd99972ce72ef7ee618806d3eab.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16)#0 (f7461d8190b6de4a64a11ea453cc24f4) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 985a3784174880eb23d851d0050907d5.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16)#0 (f7461d8190b6de4a64a11ea453cc24f4) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (1/16)#0 (d3af30b970c71ce2ebb1b7a170affb81), deploy into slot with allocation id 985a3784174880eb23d851d0050907d5.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16)#0 (f7461d8190b6de4a64a11ea453cc24f4) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16) (f7461d8190b6de4a64a11ea453cc24f4) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16)#0 (f7461d8190b6de4a64a11ea453cc24f4) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (16/16) (f7461d8190b6de4a64a11ea453cc24f4) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: DataSink (DiscardingOutput) (1/16)#0 (d3af30b970c71ce2ebb1b7a170affb81) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot bdf62dce3c1bf813d6bf1dbe4e6f07d8.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task DataSink (DiscardingOutput) (1/16)#0 (d3af30b970c71ce2ebb1b7a170affb81) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (2/16)#0 (3ec7dc737038a91397b8534d2f991d64), deploy into slot with allocation id bdf62dce3c1bf813d6bf1dbe4e6f07d8.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 90a27f0e6ea8194c8b895d13d9daaee7.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (3/16)#0 (39698f50d3dcad65d59312be8fcac566), deploy into slot with allocation id 90a27f0e6ea8194c8b895d13d9daaee7.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: DataSink (DiscardingOutput) (2/16)#0 (3ec7dc737038a91397b8534d2f991d64) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task DataSink (DiscardingOutput) (2/16)#0 (3ec7dc737038a91397b8534d2f991d64) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: DataSink (DiscardingOutput) (3/16)#0 (39698f50d3dcad65d59312be8fcac566) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 33d07cdb9708d92e2e20789d670b2c3d.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task DataSink (DiscardingOutput) (3/16)#0 (39698f50d3dcad65d59312be8fcac566) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (4/16)#0 (02543f4f1fada5007f4f4b34b3188cf4), deploy into slot with allocation id 33d07cdb9708d92e2e20789d670b2c3d.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: DataSink (DiscardingOutput) (4/16)#0 (02543f4f1fada5007f4f4b34b3188cf4) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 3d256dfcf0235812f9ce398c8f244011.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16)#0 (8f323e274877d7d2c8ff45633a2685e3) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16)#0 (8f323e274877d7d2c8ff45633a2685e3) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (5/16)#0 (54c5d96feec73c83a022b85b80d9eb12), deploy into slot with allocation id 3d256dfcf0235812f9ce398c8f244011.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16)#0 (8f323e274877d7d2c8ff45633a2685e3) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16)#0 (8f323e274877d7d2c8ff45633a2685e3) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task DataSink (DiscardingOutput) (4/16)#0 (02543f4f1fada5007f4f4b34b3188cf4) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16) (8f323e274877d7d2c8ff45633a2685e3) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Loading JAR files for task DataSink (DiscardingOutput) (4/16)#0 (02543f4f1fada5007f4f4b34b3188cf4) [DEPLOYING].'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16) (8f323e274877d7d2c8ff45633a2685e3) switched from DEPLOYING to INITIALIZING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: MapPartition (MapPartition at [1]Read with BigQuery Storage API/_PassThroughThenCleanupTempDatasets/ParDo(CleanUpProjects)) (15/16) (8f323e274877d7d2c8ff45633a2685e3) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: DataSink (DiscardingOutput) (5/16)#0 (54c5d96feec73c83a022b85b80d9eb12) switched from CREATED to DEPLOYING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl markExistingSlotActive'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Activate slot 156ecc50b0b3b22f524c9ee637e5be33.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message."
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Dec 02, 2021 6:12:00 AM org.apache.flink.runtime.taskexecutor.TaskExecutor submitTask'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Received task DataSink (DiscardingOutput) (6/16)#0 (2f26879a77baec14314f0ecd12222e8e), deploy into slot with allocation id 156ecc50b0b3b22f524c9ee637e5be33.'
FAILURE: Build failed with an exception.
* What went wrong:
GC overhead limit exceeded
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python36 - Build # 4671 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4671 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4671/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python36 #4670
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4670/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13360][Playground]
[noreply] Merge pull request #16011 from [BEAM-12164] Add models for Spanner
[noreply] [BEAM-13335] Use signed int64 range for DataFrame read unique indexes
------------------------------------------
[...truncated 47.92 MB...]
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m== 1 failed, 60 passed, 13 skipped, 173 warnings, 10 error in 6488.15 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 34s
211 actionable tasks: 153 executed, 54 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xvdurxxetiwdi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4669
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4669/display/redirect>
Changes:
------------------------------------------
[...truncated 84.68 MB...]
seconds: 1638362722
nanos: 673447132
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-12"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 706940412
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 707273721
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 707445621
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 707578659
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 708573579
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638362722
nanos: 708734989
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 51.613988637924194 seconds.
INFO:root:Successfully completed job in 51.613988637924194 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36661
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f75cdcdb400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f75cdcdb488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f75cdcdbb70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempb_uwsu6y/artifacts6gt4ljo6' '--job-port' '49239' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39597'
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44525'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:49239'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
WARNING:root:Waiting for grpc channel to be ready at localhost:49239.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a4c52982-f9bc-4371-8943-da470110e518.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a4c52982-f9bc-4371-8943-da470110e518.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a4c52982-f9bc-4371-8943-da470110e518.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a4c52982-f9bc-4371-8943-da470110e518.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:56 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1201124556-e70e609d_f7eae82c-a576-4199-a96a-11a34183c400'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:56 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1201124556-e70e609d_f7eae82c-a576-4199-a96a-11a34183c400'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:45:59 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/12/01 12:46:02 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:03 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:03 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:03 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1201124556-e70e609d_f7eae82c-a576-4199-a96a-11a34183c400 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37201.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37453.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34135
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:06 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1201124556-e70e609d_f7eae82c-a576-4199-a96a-11a34183c400: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.20 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 12:46:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1201124556-e70e609d_f7eae82c-a576-4199-a96a-11a34183c400 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638362770.548999088","description":"Error received from peer ipv4:127.0.0.1:34135","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638362770.548999088","description":"Error received from peer ipv4:127.0.0.1:34135","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638362770.549004351","description":"Error received from peer ipv4:127.0.0.1:37453","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638362770.549006585","description":"Error received from peer ipv4:127.0.0.1:37201","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 17s
211 actionable tasks: 149 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/h2msfcqxh37uw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4668
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4668/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16090: [BEAM-13352] Add transient qualifier to
------------------------------------------
[...truncated 21.34 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638339708
nanos: 990236997
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 108915805
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 136007308
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 136208295
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 136284828
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 136354207
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 136965513
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638339709
nanos: 137096405
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 14.06427526473999 seconds.
INFO:root:Successfully completed job in 14.06427526473999 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45261
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8687fe0400> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8687fe0488> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8687fe0b70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp09c7u83v/artifacts_l01oncc' '--job-port' '34775' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:34775.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45049'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41175'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34775'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:34775.
WARNING:root:Waiting for grpc channel to be ready at localhost:34775.
WARNING:root:Waiting for grpc channel to be ready at localhost:34775.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_165ad8b0-9b2c-4b42-a6a3-24f522d1689d.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_165ad8b0-9b2c-4b42-a6a3-24f522d1689d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_165ad8b0-9b2c-4b42-a6a3-24f522d1689d.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_165ad8b0-9b2c-4b42-a6a3-24f522d1689d.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:09 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1201062209-1eca731c_5c748a03-f7a9-4abe-bc53-5c615e4769ba'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1201062209-1eca731c_5c748a03-f7a9-4abe-bc53-5c615e4769ba'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:10 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:12 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1201062209-1eca731c_5c748a03-f7a9-4abe-bc53-5c615e4769ba on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:34017.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45179.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42343
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1201062209-1eca731c_5c748a03-f7a9-4abe-bc53-5c615e4769ba: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/01 06:22:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1201062209-1eca731c_5c748a03-f7a9-4abe-bc53-5c615e4769ba finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638339740.519456206","description":"Error received from peer ipv4:127.0.0.1:42343","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638339740.519476648","description":"Error received from peer ipv4:127.0.0.1:45179","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638339740.519475930","description":"Error received from peer ipv4:127.0.0.1:34017","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638339740.519456206","description":"Error received from peer ipv4:127.0.0.1:42343","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 42s
211 actionable tasks: 158 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bnrccql54tukw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python36 - Build # 4667 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4667 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4667/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python36 #4666
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4666/display/redirect?page=changes>
Changes:
[Pablo Estrada] [py] Supporting ignore_unknown_values for WriteToBigQuery
[noreply] Merge pull request #16072 from [BEAM-13135][Playground] Add function to
[noreply] Merge pull request #16084 from [BEAM-13347][Playground] [Bugfix] Backend
[noreply] Merge pull request #15987 from [BEAM-13242] Allow values with smaller
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in Extensions (#16033)
[noreply] Merge pull request #15936 from [BEAM-13351] FhirIO GetPatientEverything
[noreply] Merge pull request #16073 from [BEAM-13267][Playground] Implement
------------------------------------------
[...truncated 66.79 MB...]
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 3 failed, 60 passed, 21 skipped, 179 warnings in 5845.86 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 50m 31s
211 actionable tasks: 157 executed, 50 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/b22vhhyq63hb2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python36 - Build # 4665 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4665 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4665/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python36 #4664
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4664/display/redirect>
Changes:
------------------------------------------
[...truncated 40.84 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 41260242
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 65639495
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 65890789
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 65984487
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 66067457
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 66714286
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638253532
nanos: 66864013
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 13.989683151245117 seconds.
INFO:root:Successfully completed job in 13.989683151245117 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46273
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa1d6ee7268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa1d6ee72f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa1d6ee79d8> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempayv72dzm/artifacts8czfr6bg' '--job-port' '45735' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38827'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42365'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:45735'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a199f085-e2c9-4b69-85e3-5c296ee37f1b.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a199f085-e2c9-4b69-85e3-5c296ee37f1b.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a199f085-e2c9-4b69-85e3-5c296ee37f1b.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:37 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a199f085-e2c9-4b69-85e3-5c296ee37f1b.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:38 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1130062538-2a6671a6_df69ed14-2092-4e68-a5ff-ae85b9fe925c'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:38 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1130062538-2a6671a6_df69ed14-2092-4e68-a5ff-ae85b9fe925c'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:38 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:38 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:39 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:39 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1130062538-2a6671a6_df69ed14-2092-4e68-a5ff-ae85b9fe925c on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33033.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42373.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40235
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:41 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1130062538-2a6671a6_df69ed14-2092-4e68-a5ff-ae85b9fe925c: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.18 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/30 06:25:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1130062538-2a6671a6_df69ed14-2092-4e68-a5ff-ae85b9fe925c finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638253542.948339141","description":"Error received from peer ipv4:127.0.0.1:33033","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638253542.948312863","description":"Error received from peer ipv4:127.0.0.1:42373","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638253542.948306613","description":"Error received from peer ipv4:127.0.0.1:40235","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638253542.948306613","description":"Error received from peer ipv4:127.0.0.1:40235","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 44m 24s
211 actionable tasks: 158 executed, 49 from cache, 4 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
https://gradle.com/s/ys5wnkun3pnfk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python36 - Build # 4663 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4663 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4663/ to view the results.
beam_PostCommit_Python36 - Build # 4662 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python36 - Build # 4662 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python36/4662/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python36 #4661
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4661/display/redirect>
Changes:
------------------------------------------
[...truncated 44.13 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 274191379
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 380510091
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 392743349
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 392968893
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 393091678
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 393213033
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 394897460
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638188345
nanos: 395031690
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 21.51586079597473 seconds.
INFO:root:Successfully completed job in 21.51586079597473 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38589
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2b5038e1e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2b5038e268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2b5038e950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempg93nui4p/artifactskhws1s2a' '--job-port' '42779' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43745'
WARNING:root:Waiting for grpc channel to be ready at localhost:42779.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33143'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:42779'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:42779.
WARNING:root:Waiting for grpc channel to be ready at localhost:42779.
WARNING:root:Waiting for grpc channel to be ready at localhost:42779.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_55f6a083-340e-4cb8-ab18-560a7ec09f17.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_55f6a083-340e-4cb8-ab18-560a7ec09f17.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_55f6a083-340e-4cb8-ab18-560a7ec09f17.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:23 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_55f6a083-340e-4cb8-ab18-560a7ec09f17.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:23 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1129121923-8ffade09_651d0c8f-38e7-4e9d-a3ad-79c23c8841a4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:23 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1129121923-8ffade09_651d0c8f-38e7-4e9d-a3ad-79c23c8841a4'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:24 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:26 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:28 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:28 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1129121923-8ffade09_651d0c8f-38e7-4e9d-a3ad-79c23c8841a4 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35519.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40069.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44465
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:30 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129121923-8ffade09_651d0c8f-38e7-4e9d-a3ad-79c23c8841a4: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 12:19:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129121923-8ffade09_651d0c8f-38e7-4e9d-a3ad-79c23c8841a4 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638188372.504451953","description":"Error received from peer ipv4:127.0.0.1:35519","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638188372.504458675","description":"Error received from peer ipv4:127.0.0.1:44465","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638188372.504458675","description":"Error received from peer ipv4:127.0.0.1:44465","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638188372.504458897","description":"Error received from peer ipv4:127.0.0.1:40069","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 26s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/a73f6bokzqlhq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4660
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4660/display/redirect>
Changes:
------------------------------------------
[...truncated 45.43 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 65140247
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 181240081
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 208697557
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 208916187
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 209010362
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 209090709
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 209663629
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638166671
nanos: 209800243
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.909610748291016 seconds.
INFO:root:Successfully completed job in 5.909610748291016 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42063
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f020b8d31e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f020b8d3268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f020b8d3950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempsachabb6/artifactsbkj3ep32' '--job-port' '39503' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41979'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36967'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39503'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_743ef847-023d-41d4-af33-0c25f97e7cd7.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_743ef847-023d-41d4-af33-0c25f97e7cd7.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_743ef847-023d-41d4-af33-0c25f97e7cd7.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_743ef847-023d-41d4-af33-0c25f97e7cd7.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:57 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1129061757-29843749_cecc264c-9874-4dee-9479-e3e539e38d03'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:57 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1129061757-29843749_cecc264c-9874-4dee-9479-e3e539e38d03'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:57 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:58 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:58 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:17:58 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1129061757-29843749_cecc264c-9874-4dee-9479-e3e539e38d03 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39971.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38947.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36671
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129061757-29843749_cecc264c-9874-4dee-9479-e3e539e38d03: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 06:18:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129061757-29843749_cecc264c-9874-4dee-9479-e3e539e38d03 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638166681.773505392","description":"Error received from peer ipv4:127.0.0.1:36671","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638166681.773505392","description":"Error received from peer ipv4:127.0.0.1:36671","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638166681.773519039","description":"Error received from peer ipv4:127.0.0.1:38947","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638166681.773544013","description":"Error received from peer ipv4:127.0.0.1:39971","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 26s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pdv7slwgxsjv2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4659
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4659/display/redirect>
Changes:
------------------------------------------
[...truncated 17.89 MB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 600750446
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 729293823
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 744588613
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 744752407
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 744826316
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 744900941
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 746134281
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145536
nanos: 746234416
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 20.703060626983643 seconds.
INFO:root:Successfully completed job in 20.703060626983643 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42033
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe1825711e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe182571268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe182571950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempm9vjxn6g/artifacts7nv8ggsk' '--job-port' '58257' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:58257.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33649'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40795'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:58257'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:58257.
WARNING:root:Waiting for grpc channel to be ready at localhost:58257.
WARNING:root:Waiting for grpc channel to be ready at localhost:58257.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_698998e4-85e1-469e-8766-500c0483fad0.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_698998e4-85e1-469e-8766-500c0483fad0.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_698998e4-85e1-469e-8766-500c0483fad0.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:52 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_698998e4-85e1-469e-8766-500c0483fad0.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:53 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1129002553-ef3e4810_d7dbbe1c-3a03-4bf2-862f-69942a55bea4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:53 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1129002553-ef3e4810_d7dbbe1c-3a03-4bf2-862f-69942a55bea4'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:54 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:55 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/11/29 00:25:57 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:57 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:58 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:58 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1129002553-ef3e4810_d7dbbe1c-3a03-4bf2-862f-69942a55bea4 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:34565.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:25:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33085.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46437
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:00 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129002553-ef3e4810_d7dbbe1c-3a03-4bf2-862f-69942a55bea4: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.16 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:26:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129002553-ef3e4810_d7dbbe1c-3a03-4bf2-862f-69942a55bea4 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145561.678396749","description":"Error received from peer ipv4:127.0.0.1:34565","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145561.678369017","description":"Error received from peer ipv4:127.0.0.1:46437","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145561.678397371","description":"Error received from peer ipv4:127.0.0.1:33085","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145561.678369017","description":"Error received from peer ipv4:127.0.0.1:46437","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 13s
211 actionable tasks: 148 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fh7tnedgh3a3o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4658
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4658/display/redirect>
Changes:
------------------------------------------
[...truncated 30.56 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 175 warnings in 6664.15 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 25s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6dlydppohyc26
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4657
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4657/display/redirect>
Changes:
------------------------------------------
[...truncated 47.91 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 179 warnings in 6709.25 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 43s
211 actionable tasks: 150 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jkcpsqkyfmimq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4656
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4656/display/redirect?page=changes>
Changes:
[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting
[vitaly.ivanov] [BEAM-13269] Data exceeds database column capacity error while inserting
[noreply] Merge pull request #16074 from [BEAM-13323][Playground] [Bugfix] Fix CI
------------------------------------------
[...truncated 15.27 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 854939937
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 961280107
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 973528146
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 973695278
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 973793745
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 973913908
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 974431753
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638082205
nanos: 974545240
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 27.435611724853516 seconds.
INFO:root:Successfully completed job in 27.435611724853516 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39937
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f067e6301e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f067e630268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f067e630950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempcfiopibp/artifacts2l9qa1ce' '--job-port' '55713' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:55713.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:19 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33909'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33781'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:55713'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:55713.
WARNING:root:Waiting for grpc channel to be ready at localhost:55713.
WARNING:root:Waiting for grpc channel to be ready at localhost:55713.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a0805467-b7f0-4172-bcb3-109c57e41ce2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a0805467-b7f0-4172-bcb3-109c57e41ce2.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a0805467-b7f0-4172-bcb3-109c57e41ce2.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a0805467-b7f0-4172-bcb3-109c57e41ce2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:25 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1128065025-685778dd_10548f63-4e83-4c24-8620-c10b9b790759'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1128065025-685778dd_10548f63-4e83-4c24-8620-c10b9b790759'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:30 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1128065025-685778dd_10548f63-4e83-4c24-8620-c10b9b790759 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43187.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36995.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38093
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:32 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128065025-685778dd_10548f63-4e83-4c24-8620-c10b9b790759: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.16 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 06:50:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128065025-685778dd_10548f63-4e83-4c24-8620-c10b9b790759 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638082234.808160353","description":"Error received from peer ipv4:127.0.0.1:38093","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638082234.808160353","description":"Error received from peer ipv4:127.0.0.1:38093","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638082234.808159869","description":"Error received from peer ipv4:127.0.0.1:43187","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638082234.808159908","description":"Error received from peer ipv4:127.0.0.1:36995","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 59m 44s
211 actionable tasks: 150 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/b4eypzh6ttmjg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4655
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4655/display/redirect>
Changes:
------------------------------------------
[...truncated 43.24 MB...]
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 634374141
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 753047704
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 778846263
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 779006719
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 779080152
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 779186725
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 780870914
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058694
nanos: 780966281
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.128551959991455 seconds.
INFO:root:Successfully completed job in 6.128551959991455 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44463
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28694581e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2869458268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2869458950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp8m47ob2y/artifactsjcrqgo_q' '--job-port' '53841' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39393'
WARNING:root:Waiting for grpc channel to be ready at localhost:53841.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42087'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53841'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:53841.
WARNING:root:Waiting for grpc channel to be ready at localhost:53841.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_faa93a16-b9cd-4cf1-a380-d332f2e6ad68.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_faa93a16-b9cd-4cf1-a380-d332f2e6ad68.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_faa93a16-b9cd-4cf1-a380-d332f2e6ad68.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_faa93a16-b9cd-4cf1-a380-d332f2e6ad68.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1128001826-c7b0a62e_2ed0496b-1b0e-47e9-b845-c3c2e9669b29'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:27 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1128001826-c7b0a62e_2ed0496b-1b0e-47e9-b845-c3c2e9669b29'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:27 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:29 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:32 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1128001826-c7b0a62e_2ed0496b-1b0e-47e9-b845-c3c2e9669b29 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35357.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38293.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37599
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128001826-c7b0a62e_2ed0496b-1b0e-47e9-b845-c3c2e9669b29: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.13 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:18:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128001826-c7b0a62e_2ed0496b-1b0e-47e9-b845-c3c2e9669b29 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058715.696019303","description":"Error received from peer ipv4:127.0.0.1:37599","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058715.696041888","description":"Error received from peer ipv4:127.0.0.1:38293","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058715.696021211","description":"Error received from peer ipv4:127.0.0.1:35357","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058715.696019303","description":"Error received from peer ipv4:127.0.0.1:37599","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 44s
211 actionable tasks: 153 executed, 54 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6hbpdsddgbmcm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4654
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4654/display/redirect>
Changes:
------------------------------------------
[...truncated 31.72 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 177 warnings in 5797.34 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 43s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xryareeioxlh4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4653
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4653/display/redirect>
Changes:
------------------------------------------
[...truncated 31.79 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 187 warnings in 6581.67 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 0s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kkwyihykaisuo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4652
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4652/display/redirect>
Changes:
------------------------------------------
[...truncated 18.47 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 767308950
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 871788263
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 885695457
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 885891914
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 885978460
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 886052131
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 886660099
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993826
nanos: 886775255
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.648200035095215 seconds.
INFO:root:Successfully completed job in 6.648200035095215 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45537
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe81f4fa1e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe81f4fa268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe81f4fa950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempog9axoej/artifactst5twmpjw' '--job-port' '50973' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36763'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43111'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50973'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_96ea9027-b9f7-406c-a77e-cc86941f7864.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_96ea9027-b9f7-406c-a77e-cc86941f7864.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_96ea9027-b9f7-406c-a77e-cc86941f7864.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:12 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_96ea9027-b9f7-406c-a77e-cc86941f7864.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1127061712-20841b59_d154465c-658f-463f-8e23-9e9befad9aa8'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:13 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1127061712-20841b59_d154465c-658f-463f-8e23-9e9befad9aa8'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:13 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:13 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1127061712-20841b59_d154465c-658f-463f-8e23-9e9befad9aa8 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:46671.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35743.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33089
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127061712-20841b59_d154465c-658f-463f-8e23-9e9befad9aa8: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:17:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127061712-20841b59_d154465c-658f-463f-8e23-9e9befad9aa8 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993837.092523760","description":"Error received from peer ipv4:127.0.0.1:46671","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993837.092491762","description":"Error received from peer ipv4:127.0.0.1:33089","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993837.092523741","description":"Error received from peer ipv4:127.0.0.1:35743","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993837.092491762","description":"Error received from peer ipv4:127.0.0.1:33089","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py36:postCommitPy36
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 1s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/k7lhdrc6z5wdg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4651
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4651/display/redirect>
Changes:
------------------------------------------
[...truncated 11.67 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (2/16) (f9f2371a1ba8a3c443f489810a9c33f4) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (9/16)#0 d5aeb99ad8bd36d7c1c86f23b35bdb92.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (10/16)#0 ca8375700c295f660a9ec3d6b8add815.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (2/16)#0 8d38b4af75fc6542a2dd4e34b3202e71.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (1/16)#0 feb2bf3e21d88cabfb2bfb12f45963ac.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (13/16)#0 48f2910ec7659f82ac3b2165531528c5.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (12/16)#0 b286462fec7d9616713cc8eb0b7d8525.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (13/16)#0 62a597d9b7b4e21728786863d1fd9c4d.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (8/16)#0 d5d0ac2ff7225de8ed18b12b624155c6.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (4/16)#0 370c78f54ceb679bdaf2f308bad87b0e.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (7/16)#0 68778e19ae0c1f1b5651eee9d7677c10.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (3/16)#0 0e992cf6c9392cb89a6f29e6ceccbad5.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task ToKeyedWorkItem (15/16)#0 acd39710c73ffc6f718cdc2cc9c9f03e.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (12/16) (3736a9666ae991d7e2682a07d2449c3e) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (3/16) (ef7487aef2fc5ad4dbd424f628120794) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (15/16) (e66c17b087585b20e98d74b18c82029c) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (10/16) (fabf1bb0e0ead0d7610515b7e0c11f6d) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (1/16) (60270084f06b813e79e715a0dcddc849) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (7/16) (390d56083ec24131aef5ca9e170e9d0c) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (4/16) (246a1c3c4a122a34272647b966179359) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (9/16) (959382cc963ff2df6902747b286b942a) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (8/16) (91e2dea2a9078e393eb670660e589d3c) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (9/16) (d5aeb99ad8bd36d7c1c86f23b35bdb92) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (10/16) (ca8375700c295f660a9ec3d6b8add815) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (2/16) (8d38b4af75fc6542a2dd4e34b3202e71) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (1/16) (feb2bf3e21d88cabfb2bfb12f45963ac) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (13/16) (48f2910ec7659f82ac3b2165531528c5) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (12/16) (b286462fec7d9616713cc8eb0b7d8525) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (13/16) (62a597d9b7b4e21728786863d1fd9c4d) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (8/16) (d5d0ac2ff7225de8ed18b12b624155c6) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (4/16) (370c78f54ceb679bdaf2f308bad87b0e) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (7/16) (68778e19ae0c1f1b5651eee9d7677c10) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (3/16) (0e992cf6c9392cb89a6f29e6ceccbad5) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: ToKeyedWorkItem (15/16) (acd39710c73ffc6f718cdc2cc9c9f03e) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Starting PostgresConnectorTask with configuration:'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: connector.class = io.debezium.connector.postgresql.PostgresConnector'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.dbname = inventory'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.user = debezium'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.hostname = localhost'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.password = ********'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: beam.parent.instance = 324575553'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.server.name = dbserver1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.history = org.apache.beam.io.debezium.KafkaSourceConsumerFn$DebeziumSDFDatabaseHistory'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: include.schema.changes = false'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.include.list = inventory'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: database.port = 33540'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT
[gw6] [32mPASSED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/io/fileio_test.py::MatchIntegrationTest::test_transform_on_gcs
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: user 'debezium' connected to database 'inventory' on PostgreSQL 11.14 (Debian 11.14-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit with roles:"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_all_settings' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_stat_scan_tables' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_write_server_files' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'debezium' [superuser: true, replication: true, inherit: true, create role: true, create db: true, can log in: true]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_monitor' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_server_files' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_execute_server_program' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_all_stats' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_signal_backend' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Obtained valid replication slot ReplicationSlot [active=false, latestFlushedLsn=null, catalogXmin=null]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: No previous offset found'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Taking initial snapshot for new datasource'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Creating replication slot with command CREATE_REPLICATION_SLOT debezium LOGICAL decoderbufs'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Requested thread factory for connector PostgresConnector, id = dbserver1 named = change-event-source-coordinator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Creating thread debezium-postgresconnector-dbserver1-change-event-source-coordinator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Metrics registered'
> Task :sdks:python:test-suites:portable:py36:postCommitPy36IT
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.blacklist" is deprecated and will be removed in future versions. Please use "schema.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.blacklist" is deprecated and will be removed in future versions. Please use "table.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "table.whitelist" is deprecated and will be removed in future versions. Please use "table.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "schema.whitelist" is deprecated and will be removed in future versions. Please use "schema.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.whitelist" is deprecated and will be removed in future versions. Please use "column.include.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'WARNING: Using configuration property "column.blacklist" is deprecated and will be removed in future versions. Please use "column.exclude.list" instead.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"INFO: user 'debezium' connected to database 'inventory' on PostgreSQL 11.14 (Debian 11.14-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit with roles:"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_all_settings' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_stat_scan_tables' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_write_server_files' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'debezium' [superuser: true, replication: true, inherit: true, create role: true, create db: true, can log in: true]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_monitor' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_server_files' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_execute_server_program' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_read_all_stats' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b"\trole 'pg_signal_backend' [superuser: false, replication: false, inherit: true, create role: false, create db: false, can log in: false]"
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Obtained valid replication slot ReplicationSlot [active=false, latestFlushedLsn=null, catalogXmin=null]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: No previous offset found'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Taking initial snapshot for new datasource'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:40 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Creating replication slot with command CREATE_REPLICATION_SLOT debezium LOGICAL decoderbufs'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Requested thread factory for connector PostgresConnector, id = dbserver1 named = change-event-source-coordinator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Creating thread debezium-postgresconnector-dbserver1-change-event-source-coordinator'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'java.lang.OutOfMemoryError: GC overhead limit exceeded
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Metrics registered'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Context created'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Taking initial snapshot for new datasource'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: According to the connector configuration data will be snapshotted'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 27, 2021 12:27:41 AM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4650
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4650/display/redirect>
Changes:
------------------------------------------
[...truncated 31.70 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 173 warnings in 6609.47 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 1s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/n3ariemi3vke2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4649
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4649/display/redirect>
Changes:
------------------------------------------
[...truncated 56.37 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 493852376
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 521857023
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 522095918
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 522232294
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 522357225
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 523054361
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637929416
nanos: 523239135
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 16.34241819381714 seconds.
INFO:root:Successfully completed job in 16.34241819381714 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37669
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa6221071e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa622107268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa622107950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp_6hdecly/artifactsef8da29d' '--job-port' '53229' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:53229.
WARNING:root:Waiting for grpc channel to be ready at localhost:53229.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39847'
WARNING:root:Waiting for grpc channel to be ready at localhost:53229.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42123'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:53229'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:53 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:53229.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_3da72773-065e-44ef-ad60-a99e9f7dc591.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_3da72773-065e-44ef-ad60-a99e9f7dc591.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_3da72773-065e-44ef-ad60-a99e9f7dc591.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_3da72773-065e-44ef-ad60-a99e9f7dc591.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:57 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1126122357-44221b5a_eb12ae7c-1339-4368-b5ed-e5992d0fcfa0'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:57 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1126122357-44221b5a_eb12ae7c-1339-4368-b5ed-e5992d0fcfa0'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:23:58 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:00 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/11/26 12:24:03 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1126122357-44221b5a_eb12ae7c-1339-4368-b5ed-e5992d0fcfa0 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:34847.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35841.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41985
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:06 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:07 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126122357-44221b5a_eb12ae7c-1339-4368-b5ed-e5992d0fcfa0: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.11 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 12:24:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126122357-44221b5a_eb12ae7c-1339-4368-b5ed-e5992d0fcfa0 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637929448.886053168","description":"Error received from peer ipv4:127.0.0.1:41985","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637929448.886053168","description":"Error received from peer ipv4:127.0.0.1:41985","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637929448.886078919","description":"Error received from peer ipv4:127.0.0.1:34847","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637929448.886053087","description":"Error received from peer ipv4:127.0.0.1:35841","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 13s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6gxlb5m62xo6y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4648
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4648/display/redirect>
Changes:
------------------------------------------
[...truncated 31.70 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 177 warnings in 5751.47 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 10s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kt6bklhgogsk2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4647
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4647/display/redirect>
Changes:
------------------------------------------
[...truncated 32.08 MB...]
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 111080408
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 118428230
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 118623971
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 118762493
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 118905067
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 120797634
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637886538
nanos: 120984077
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.149080276489258 seconds.
INFO:root:Successfully completed job in 11.149080276489258 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38651
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7d82b121e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7d82b12268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7d82b12950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempz0yh3xjs/artifactspmwxwduu' '--job-port' '40967' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:40967.
WARNING:root:Waiting for grpc channel to be ready at localhost:40967.
WARNING:root:Waiting for grpc channel to be ready at localhost:40967.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46417'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:10 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35529'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:10 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40967'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:10 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:40967.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:13 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a9083601-a133-4b59-abd3-4d95be949dbe.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:13 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a9083601-a133-4b59-abd3-4d95be949dbe.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:13 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a9083601-a133-4b59-abd3-4d95be949dbe.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:13 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a9083601-a133-4b59-abd3-4d95be949dbe.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:14 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1126002914-b44448f_92a8ff3b-b2fa-4f9d-96d1-e1b42ae27582'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:14 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1126002914-b44448f_92a8ff3b-b2fa-4f9d-96d1-e1b42ae27582'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:15 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:17 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:22 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:22 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1126002914-b44448f_92a8ff3b-b2fa-4f9d-96d1-e1b42ae27582 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44093.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39853.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36947
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:24 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126002914-b44448f_92a8ff3b-b2fa-4f9d-96d1-e1b42ae27582: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 00:29:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126002914-b44448f_92a8ff3b-b2fa-4f9d-96d1-e1b42ae27582 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637886567.671492807","description":"Error received from peer ipv4:127.0.0.1:36947","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637886567.671492807","description":"Error received from peer ipv4:127.0.0.1:36947","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637886567.674297961","description":"Error received from peer ipv4:127.0.0.1:39853","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637886567.674322181","description":"Error received from peer ipv4:127.0.0.1:44093","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 24s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6nijejrsvzbya
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4646
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4646/display/redirect>
Changes:
------------------------------------------
[...truncated 56.12 MB...]
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 547430515
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 575750350
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 575937747
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 576010942
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 576079130
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 576604604
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637864607
nanos: 576721191
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 9.441797971725464 seconds.
INFO:root:Successfully completed job in 9.441797971725464 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py36:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41205
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9503c531e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9503c53268> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9503c53950> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temppmidex9x/artifactsu959ymbx' '--job-port' '36315' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:36315.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36967'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43595'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36315'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:36315.
WARNING:root:Waiting for grpc channel to be ready at localhost:36315.
WARNING:root:Waiting for grpc channel to be ready at localhost:36315.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a9d7b0d4-af78-4852-8908-bfc5f2689631.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a9d7b0d4-af78-4852-8908-bfc5f2689631.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a9d7b0d4-af78-4852-8908-bfc5f2689631.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a9d7b0d4-af78-4852-8908-bfc5f2689631.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1125182341-8e6a5954_7c3944e8-e1dc-444c-af6a-c8fd0b13b0ac'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:41 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1125182341-8e6a5954_7c3944e8-e1dc-444c-af6a-c8fd0b13b0ac'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:42 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:44 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:47 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:47 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1125182341-8e6a5954_7c3944e8-e1dc-444c-af6a-c8fd0b13b0ac on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33989.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35681.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34587
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1125182341-8e6a5954_7c3944e8-e1dc-444c-af6a-c8fd0b13b0ac: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 18:23:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1125182341-8e6a5954_7c3944e8-e1dc-444c-af6a-c8fd0b13b0ac finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637864631.320261255","description":"Error received from peer ipv4:127.0.0.1:35681","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637864631.320247546","description":"Error received from peer ipv4:127.0.0.1:34587","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637864631.320270184","description":"Error received from peer ipv4:127.0.0.1:33989","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/build/gradleenv/2022703440/lib/python3.6/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637864631.320247546","description":"Error received from peer ipv4:127.0.0.1:34587","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 47s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ynzrmbihnxzhu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4645
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4645/display/redirect>
Changes:
------------------------------------------
[...truncated 48.00 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 177 warnings in 6641.52 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 42s
211 actionable tasks: 147 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/f6z3fgahetlma
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python36 #4644
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python36/4644/display/redirect>
Changes:
------------------------------------------
[...truncated 44.22 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/pytest_postCommitIT-df-py36.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 175 warnings in 6861.92 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py36:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py36:postCommitPy36IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python36/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py36:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 9s
211 actionable tasks: 155 executed, 52 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5usujzutewgmm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org