You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/11/04 13:43:57 UTC
Build failed in Jenkins: beam_PostCommit_Python37 #4475
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4475/display/redirect>
Changes:
------------------------------------------
[...truncated 48.17 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============ 63 passed, 21 skipped, 177 warnings in 5857.24 seconds ============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 43m 20s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/w7sd4r7z2n2mo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4616
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4616/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11936] Remove suppression in ReadFromKafkaDoFn (#16174)
[noreply] [BEAM-12561] method truncate on series and dataframe (#15833)
------------------------------------------
[...truncated 34.34 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PTRANSFORM": "fn/read/pcollection_1:0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:element_count:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:sum_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "AQ==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "pcollection_2"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "Ac0TzRPNEw==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "pcollection_2"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "Ac4TzhPOEw==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "external_7Write to Spanner/SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/CombineValues/Values/Values/Map/ParMultiDo(Anonymous).output"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "AdQT1BPUEw==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "pcollection_1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "AcwTzBPMEw==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "external_7Write to Spanner/SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous).output"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:2, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: a25f80e10ab570f0e796b3e5a710609f, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:3, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 003e998e9725ef90c50ccabe6140b070, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:13, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: bf662928533097fcce5b150b519b8ace, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:9, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 76242037fd1933108d500a086772bf58, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:4, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: c531fce9aaf1eaf0fdceed8d276afdb9, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.TaskExecutor$JobLeaderListenerImpl jobManagerLostLeadership'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: JobManager for job 2a6f21228f2f5913ea191f87c6473009 with leader id 9f73a6e09b96b7a6568ea2b434db49f8 lost leadership.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:15, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: b6b47a789d18f8622b85e5772950afbd, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:7, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 1fa26c86da975bb097e2bfddd2b22b9e, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:5, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 7f075ed08875b1e16b7634ed426d4ce1, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:0, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 76558f0b42afb85fa80a0e33710ccc82, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:1, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: e55ceaef25d52250c7c0d7ae636299e0, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:8, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 8e56abaca8a99bd14c42def244e694cf, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint lambda$shutDownInternal$5'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Removing cache directory /tmp/flink-web-ui'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:6, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: ff9f630f64567c2b207727d1b57ecdca, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:11, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 6118459e11562236a785ed371eeb3580, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl freeSlotInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Free slot TaskSlot(index:12, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: b7b12f0ab58f4d11d17c899da5a194ba, jobId: 2a6f21228f2f5913ea191f87c6473009).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.rest.RestServerEndpoint lambda$closeAsync$1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shut down complete.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.resourcemanager.ResourceManager deregisterApplication'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shut down cluster because application is in CANCELED, diagnostics DispatcherResourceManagerComponent has been closed..'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent closeAsyncInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Closing components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess closeInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping SessionDispatcherLeaderProcess.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.dispatcher.Dispatcher onStop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.dispatcher.Dispatcher terminateRunningJobs'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping all currently running jobs of dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Closing the slot manager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager suspend'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Suspending the slot manager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.dispatcher.Dispatcher lambda$onStop$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stop job leader service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down TaskExecutorLocalStateStoresManager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: FileChannelManager removed spill file directory /tmp/flink-io-34600400-c0bb-4d8e-9a4c-baef0415f5c3'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.io.network.NettyShuffleEnvironment close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down the network environment and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: FileChannelManager removed spill file directory /tmp/flink-netty-shuffle-a5b920ab-1c73-49fa-8b5e-2856abead93f'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.KvStateService shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down the kvState service and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stop job leader service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.filecache.FileCache shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: removed file cache directory /tmp/flink-dist-cache-682693e9-f521-4df9-9477-914694d8c7a9'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.taskexecutor.TaskExecutor handleOnStopException'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped TaskExecutor akka://flink/user/rpc/taskmanager_0.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.blob.BlobServer close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped BLOB server at 0.0.0.0:34327'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 09, 2021 6:19:57 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped Akka RPC service.'
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:576 Job state changed to DONE
PASSED [100%]
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-flink-py37.xml> -
========================== 7 passed in 248.84 seconds ==========================
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1613.35 seconds =============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 18m 26s
217 actionable tasks: 156 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4thvrps64fcs6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4615
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4615/display/redirect>
Changes:
------------------------------------------
[...truncated 48.20 MB...]
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 181 warnings in 6192.84 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1568.27 seconds =============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 14m 41s
217 actionable tasks: 174 executed, 39 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ryoyggahkkx5i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4614
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4614/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13015] Start integrating a process wide cache. (#16130)
------------------------------------------
[...truncated 51.12 MB...]
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 187 warnings in 6278.04 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1585.80 seconds =============[0m
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 285
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 17m 53s
217 actionable tasks: 174 executed, 39 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fmciccfzskstg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4613
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4613/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11936] Remove suppressUnusedVariable flag (#16171)
[noreply] [BEAM-13090] Adding SDK harness container overrides option to Java SDK
[noreply] [BEAM-11936] Fix errorprone warnings (#15890)
------------------------------------------
[...truncated 29.10 MB...]
seconds: 1639009441
nanos: 63503265
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 76503276
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 76662778
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 76746702
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 76814889
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 77385187
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1639009441
nanos: 77491760
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.390655755996704 seconds.
INFO:root:Successfully completed job in 7.390655755996704 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42683
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fa59fb8bb90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fa59fb8bc20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fa59fb8c3b0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempq9ubw8mq/artifactslf3afckj' '--job-port' '60579' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35861'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43055'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60579'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:05 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_08478619-c085-423b-b360-9db8c6e03a63.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_08478619-c085-423b-b360-9db8c6e03a63.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_08478619-c085-423b-b360-9db8c6e03a63.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_08478619-c085-423b-b360-9db8c6e03a63.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1209002406-f893315a_f66c6b07-3bbf-40f1-b189-b0afac556e81'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:06 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1209002406-f893315a_f66c6b07-3bbf-40f1-b189-b0afac556e81'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:07 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:07 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:08 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:08 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1209002406-f893315a_f66c6b07-3bbf-40f1-b189-b0afac556e81 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45657.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:32977.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45219
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209002406-f893315a_f66c6b07-3bbf-40f1-b189-b0afac556e81: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/09 00:24:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1209002406-f893315a_f66c6b07-3bbf-40f1-b189-b0afac556e81 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639009450.708398935","description":"Error received from peer ipv4:127.0.0.1:45219","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639009450.708428288","description":"Error received from peer ipv4:127.0.0.1:32977","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639009450.708398935","description":"Error received from peer ipv4:127.0.0.1:45219","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1639009450.708446902","description":"Error received from peer ipv4:127.0.0.1:45657","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:portable:py37:xlangSpannerIOIT
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1639009681.981044082","description":"Error received from peer ipv4:127.0.0.1:42231","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 25m 33s
217 actionable tasks: 180 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/6pnruq32gl6hu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4612
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4612/display/redirect?page=changes>
Changes:
[stranniknm] fix playground frontend licences
[noreply] Merge pull request #16167 from [BEAM-13409][Playground] [Bugfix] Change
[noreply] Merge pull request #16136 from [BEAM-13365] [Playground] Add Pipelines
[noreply] [BEAM-13244] Support STS Assume role credentials provider for AWS SDK v2
[noreply] [BEAM-11936] Fix errorprone UnusedVariable in core,examples,harness..
------------------------------------------
[...truncated 48.06 MB...]
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 185 warnings in 6257.62 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
[gw4] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw4] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw4] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw4] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw4] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1654.20 seconds =============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 15m 52s
217 actionable tasks: 156 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/c7hpnxht2dxia
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4611
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4611/display/redirect?page=changes>
Changes:
[heejong] [BEAM-13092] Adding dummy external transform translators for Dataflow
[melissapa] [BEAM-11758] Final cleanup for Beam Basics doc content
------------------------------------------
[...truncated 45.78 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }, {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "urn": "beam:metric:sampled_byte_size:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "type": "beam:metrics:distribution_int64:v1",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "payload": "AcwTzBPMEw==",'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "labels": {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' "PCOLLECTION": "external_7Write to Spanner/SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/CombineValues/Combine.perKey(Singleton)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous).output"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b' }'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.webmonitor.WebMonitorEndpoint lambda$shutDownInternal$5'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Removing cache directory /tmp/flink-web-ui'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.rest.RestServerEndpoint lambda$closeAsync$1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shut down complete.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.resourcemanager.ResourceManager deregisterApplication'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shut down cluster because application is in CANCELED, diagnostics DispatcherResourceManagerComponent has been closed..'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent closeAsyncInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Closing components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stop job leader service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.taskexecutor.TaskExecutor$JobLeaderListenerImpl jobManagerLostLeadership'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: JobManager for job 04ee1005430966ee905b3af036e93e57 with leader id 8feca3304c4c43f9289b2899d7af47e9 lost leadership.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down TaskExecutorLocalStateStoresManager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess closeInternal'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping SessionDispatcherLeaderProcess.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.dispatcher.Dispatcher onStop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.dispatcher.Dispatcher terminateRunningJobs'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping all currently running jobs of dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Closing the slot manager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager suspend'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Suspending the slot manager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.dispatcher.Dispatcher lambda$onStop$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped dispatcher akka://flink/user/rpc/dispatcher_2.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: FileChannelManager removed spill file directory /tmp/flink-io-9485cbf2-509a-40a3-b32f-8cd619206deb'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.io.network.NettyShuffleEnvironment close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down the network environment and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.io.disk.FileChannelManagerImpl lambda$getFileCloser$0'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: FileChannelManager removed spill file directory /tmp/flink-netty-shuffle-6b227766-50fb-42e7-a366-42b4714b175a'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.taskexecutor.KvStateService shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down the kvState service and its components.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stop job leader service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.filecache.FileCache shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: removed file cache directory /tmp/flink-dist-cache-dec7fd45-bc70-40a3-ba65-1a6982a81a82'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.taskexecutor.TaskExecutor handleOnStopException'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped TaskExecutor akka://flink/user/rpc/taskmanager_0.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService stopService'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopping Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped Akka RPC service.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.blob.BlobServer close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped BLOB server at 0.0.0.0:38791'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Dec 08, 2021 12:22:40 PM org.apache.flink.runtime.rpc.akka.AkkaRpcService lambda$stopService$7'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: Stopped Akka RPC service.'
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:576 Job state changed to DONE
PASSED [100%]
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-flink-py37.xml> -
========================== 7 passed in 326.81 seconds ==========================
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638965982.022806071","description":"Error received from peer ipv4:127.0.0.1:44539","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638966025.109488236","description":"Error received from peer ipv4:127.0.0.1:40537","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638966127.130627295","description":"Error received from peer ipv4:127.0.0.1:38603","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1566.46 seconds =============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 16m 48s
217 actionable tasks: 156 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7xtqkcwmtgn7k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4610
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4610/display/redirect?page=changes>
Changes:
[relax] don't store unserializable DatasetService as a member variable
[dpcollins] Add a workaround for https://github.com/googleapis/gax-java/issues/1577
[noreply] [BEAM-13388] Use 3.0.0 as lower bound for google-cloud-dlp (#16164)
[noreply] Merge pull request #16127 from [BEAM-13366] [Playground] Add support
------------------------------------------
[...truncated 27.83 MB...]
seconds: 1638944412
nanos: 170340776
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 192270517
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 192482233
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 192597150
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 192699432
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 193903207
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638944412
nanos: 194066524
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.195748805999756 seconds.
INFO:root:Successfully completed job in 6.195748805999756 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34563
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2cc21f3b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2cc21f3c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2cc21f43b0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempz2g12zyu/artifactspcit3x6i' '--job-port' '46841' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34085'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41841'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:46841'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_6e3975a1-8fd8-4f83-bc58-b3b77b33a170.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_6e3975a1-8fd8-4f83-bc58-b3b77b33a170.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_6e3975a1-8fd8-4f83-bc58-b3b77b33a170.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_6e3975a1-8fd8-4f83-bc58-b3b77b33a170.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1208062017-f73b59a_1b63af8f-a5d0-4534-bde2-6133f1e83b0a'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:18 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1208062017-f73b59a_1b63af8f-a5d0-4534-bde2-6133f1e83b0a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:19 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1208062017-f73b59a_1b63af8f-a5d0-4534-bde2-6133f1e83b0a on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41973.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43589.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36441
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208062017-f73b59a_1b63af8f-a5d0-4534-bde2-6133f1e83b0a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/08 06:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1208062017-f73b59a_1b63af8f-a5d0-4534-bde2-6133f1e83b0a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944422.361165228","description":"Error received from peer ipv4:127.0.0.1:36441","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944422.361165228","description":"Error received from peer ipv4:127.0.0.1:36441","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944422.361208082","description":"Error received from peer ipv4:127.0.0.1:43589","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638944422.361232093","description":"Error received from peer ipv4:127.0.0.1:41973","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:portable:py37:xlangSpannerIOIT
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1638944533.674002275","description":"Error received from peer ipv4:127.0.0.1:45829","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 19m 34s
217 actionable tasks: 174 executed, 39 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7tbkfz3xovtrs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4609
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4609/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-13388] Fix broken google cloud dlp test.
[zyichi] [BEAM-13373] Increase python post commit timeout to reduce chance of
[msbukal] Exclude FhirIOPatientEverything from v2 dataflow runner intg test.
[noreply] [BEAM-13371] Fix bug where DataFrame overview snippets don't show up
[noreply] [BEAM-12976] Implement pipeline visitor to get global field access in…
------------------------------------------
[...truncated 48.22 MB...]
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 55 passed, 11 skipped, 179 warnings in 6727.41 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT
>>> RUNNING integration tests with pipeline options: --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/build/apache-beam.tar.gz> --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --dataflow_worker_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.36.0-SNAPSHOT.jar> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
>>> pytest options: --capture=no --timeout=4500 --color=yes --log-cli-level=INFO --numprocesses=8 --dist=loadfile
>>> collect markers: -m=spannerio_it
[1m============================= test session starts ==============================[0m
platform linux -- Python 3.7.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python,> inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[1m[gw0] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw1] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw2] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw3] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw4] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw5] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw6] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0m[1m[gw7] Python 3.7.10 (default, Feb 20 2021, 21:21:24) -- [GCC 5.4.0 20160609]
[0mgw0 [15] / gw1 [15] / gw2 [15] / gw3 [15] / gw4 [15] / gw5 [15] / gw6 [15] / gw7 [15]
scheduling tests via LoadFileScheduling
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
[gw1] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_error
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_sql
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_spanner_update
apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[gw0] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_read_via_table
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_table_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_sql_metrics_ok_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_error_call
apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw0] [33mSKIPPED[0m apache_beam/io/gcp/experimental/spannerio_read_it_test.py::SpannerReadIntegrationTest::test_transaction_table_metrics_ok_call
[gw1] [32mPASSED[0m apache_beam/io/gcp/experimental/spannerio_write_it_test.py::SpannerWriteIntegrationTest::test_write_batches
[33m=============================== warnings summary ===============================[0m
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/_pytest/mark/structures.py>:335: PytestUnknownMarkWarning: Unknown pytest.mark.spannerio_it - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
PytestUnknownMarkWarning,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[33m[1m============= 5 passed, 10 skipped, 13 warnings in 1579.76 seconds =============[0m
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 35m 14s
217 actionable tasks: 181 executed, 32 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wqodjfmrzzw6c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4608 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4608 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4608/ to view the results.
beam_PostCommit_Python37 - Build # 4607 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4607 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4607/ to view the results.
beam_PostCommit_Python37 - Build # 4606 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4606 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4606/ to view the results.
beam_PostCommit_Python37 - Build # 4605 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4605 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4605/ to view the results.
beam_PostCommit_Python37 - Build # 4604 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4604 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4604/ to view the results.
beam_PostCommit_Python37 - Build # 4603 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4603 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4603/ to view the results.
beam_PostCommit_Python37 - Build # 4602 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4602 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4602/ to view the results.
beam_PostCommit_Python37 - Build # 4601 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4601 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4601/ to view the results.
beam_PostCommit_Python37 - Build # 4600 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4600 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4600/ to view the results.
beam_PostCommit_Python37 - Build # 4599 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4599 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4599/ to view the results.
beam_PostCommit_Python37 - Build # 4598 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4598 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4598/ to view the results.
beam_PostCommit_Python37 - Build # 4597 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4597 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4597/ to view the results.
beam_PostCommit_Python37 - Build # 4596 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4596 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4596/ to view the results.
beam_PostCommit_Python37 - Build # 4595 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4595 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4595/ to view the results.
beam_PostCommit_Python37 - Build # 4594 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4594 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4594/ to view the results.
beam_PostCommit_Python37 - Build # 4593 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4593 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4593/ to view the results.
beam_PostCommit_Python37 - Build # 4592 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4592 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4592/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4591
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4591/display/redirect>
Changes:
------------------------------------------
[...truncated 29.37 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 491827726
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 595997810
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 609917640
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 610096216
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 610203742
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 610302209
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 611681699
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638533840
nanos: 611850500
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.600504398345947 seconds.
INFO:root:Successfully completed job in 5.600504398345947 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35503
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3f3a856560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3f3a8565f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3f3a856d40> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-templa0sijkv/artifacts158tb3g8' '--job-port' '48921' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33437'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42197'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:48921'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d7ba9b59-90cc-4888-a84c-67e2cc84a020.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d7ba9b59-90cc-4888-a84c-67e2cc84a020.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d7ba9b59-90cc-4888-a84c-67e2cc84a020.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d7ba9b59-90cc-4888-a84c-67e2cc84a020.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1203121726-27630355_23a33749-60bd-447e-8ce9-8eba5e23fd37'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1203121726-27630355_23a33749-60bd-447e-8ce9-8eba5e23fd37'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:27 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:28 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:28 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1203121726-27630355_23a33749-60bd-447e-8ce9-8eba5e23fd37 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35123.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34093.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36141
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203121726-27630355_23a33749-60bd-447e-8ce9-8eba5e23fd37: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 12:17:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203121726-27630355_23a33749-60bd-447e-8ce9-8eba5e23fd37 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638533850.786306723","description":"Error received from peer ipv4:127.0.0.1:36141","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638533850.786350115","description":"Error received from peer ipv4:127.0.0.1:35123","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638533850.786306723","description":"Error received from peer ipv4:127.0.0.1:36141","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638533850.786327453","description":"Error received from peer ipv4:127.0.0.1:34093","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 45m 31s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/72szcvjlppf4g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4590
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4590/display/redirect>
Changes:
------------------------------------------
[...truncated 39.94 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 231533527
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 336037635
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 350361585
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 350517511
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 350593090
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 350655555
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 351114034
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638512445
nanos: 351233959
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 4.989459753036499 seconds.
INFO:root:Successfully completed job in 4.989459753036499 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40267
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2a3586d560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2a3586d5f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2a3586dd40> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempjtcpu_t6/artifactsj6vbrsht' '--job-port' '56405' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40449'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44539'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56405'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c6ba883d-89cc-467d-9e0a-6724537463e9.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c6ba883d-89cc-467d-9e0a-6724537463e9.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_c6ba883d-89cc-467d-9e0a-6724537463e9.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c6ba883d-89cc-467d-9e0a-6724537463e9.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:50 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1203062050-32c2f975_8046017e-d4c5-442d-94c7-0912ead6e4e1'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:51 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1203062050-32c2f975_8046017e-d4c5-442d-94c7-0912ead6e4e1'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:51 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:51 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:52 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:52 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1203062050-32c2f975_8046017e-d4c5-442d-94c7-0912ead6e4e1 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37531.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43649.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38343
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203062050-32c2f975_8046017e-d4c5-442d-94c7-0912ead6e4e1: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 06:20:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203062050-32c2f975_8046017e-d4c5-442d-94c7-0912ead6e4e1 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638512455.075580960","description":"Error received from peer ipv4:127.0.0.1:38343","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638512455.075590192","description":"Error received from peer ipv4:127.0.0.1:43649","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638512455.075603006","description":"Error received from peer ipv4:127.0.0.1:37531","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638512455.075580960","description":"Error received from peer ipv4:127.0.0.1:38343","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 33s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ktf3hqz57o7ni
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4589
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4589/display/redirect?page=changes>
Changes:
[noreply] Don't pin a particular version of Tensorflow. (#16102)
[noreply] [BEAM-12733] Fix failing integration tests for Java Recommendation AI
[noreply] [BEAM-13288] improve logging for no rows present error (#16096)
------------------------------------------
[...truncated 32.09 MB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638491043
nanos: 928155183
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-12"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 55074930
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-12"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 74545860
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 74888706
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 75055122
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 75174093
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 77136754
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638491044
nanos: 77313423
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.792866945266724 seconds.
INFO:root:Successfully completed job in 11.792866945266724 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44147
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f7aeb9b13b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f7aeb9b1440> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f7aeb9b1b90> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempqxhisdht/artifactsd4084etz' '--job-port' '37523' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:37523.
WARNING:root:Waiting for grpc channel to be ready at localhost:37523.
WARNING:root:Waiting for grpc channel to be ready at localhost:37523.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:17 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45151'
WARNING:root:Waiting for grpc channel to be ready at localhost:37523.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:17 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40569'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:37523'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:18 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:19 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_cf372475-812e-4783-8015-03bef4ecb3de.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:19 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_cf372475-812e-4783-8015-03bef4ecb3de.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:19 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_cf372475-812e-4783-8015-03bef4ecb3de.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:19 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_cf372475-812e-4783-8015-03bef4ecb3de.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1203002420-6da5db0d_f9b7e4d7-318b-412f-affc-ba97577f3514'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:21 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1203002420-6da5db0d_f9b7e4d7-318b-412f-affc-ba97577f3514'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:22 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:24 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/12/03 00:24:27 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:28 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:28 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1203002420-6da5db0d_f9b7e4d7-318b-412f-affc-ba97577f3514 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:46765.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44397.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40991
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203002420-6da5db0d_f9b7e4d7-318b-412f-affc-ba97577f3514: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/03 00:24:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1203002420-6da5db0d_f9b7e4d7-318b-412f-affc-ba97577f3514 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638491075.170495783","description":"Error received from peer ipv4:127.0.0.1:40991","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638491075.170495783","description":"Error received from peer ipv4:127.0.0.1:40991","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638491075.171198185","description":"Error received from peer ipv4:127.0.0.1:46765","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638491075.170929878","description":"Error received from peer ipv4:127.0.0.1:44397","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 52s
214 actionable tasks: 156 executed, 54 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rmnntxycjdnlm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4588
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4588/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13329][Playground]
[aydar.zaynutdinov] [BEAM-13329][Playground]
[alexander.zhuravlev] [BEAM-13370] Deleted unused prints & strings
[ilya.kozyrev] Fix pylint issues and apply yapf with Beam config
[ilya.kozyrev] fix white spaces
------------------------------------------
[...truncated 2.85 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 950649023
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 974383354
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 974524259
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 974602937
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 974685192
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 975199222
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638469192
nanos: 975308179
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.408763408660889 seconds.
INFO:root:Successfully completed job in 6.408763408660889 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39667
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd1d62c8290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd1d62c8320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd1d62c8a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp5oodcmzp/artifactsrgfmuq23' '--job-port' '34465' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:34465.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33985'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33663'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34465'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:00 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:34465.
WARNING:root:Waiting for grpc channel to be ready at localhost:34465.
WARNING:root:Waiting for grpc channel to be ready at localhost:34465.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_eacadfc8-ac9e-4661-90cd-65833482eccf.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_eacadfc8-ac9e-4661-90cd-65833482eccf.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_eacadfc8-ac9e-4661-90cd-65833482eccf.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:04 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_eacadfc8-ac9e-4661-90cd-65833482eccf.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:05 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1202182005-c428410c_37e97751-c768-4104-baa3-699504f452e6'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:05 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1202182005-c428410c_37e97751-c768-4104-baa3-699504f452e6'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:06 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:08 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:11 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:11 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:11 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1202182005-c428410c_37e97751-c768-4104-baa3-699504f452e6 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35481.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40755.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43597
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202182005-c428410c_37e97751-c768-4104-baa3-699504f452e6: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.11 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 18:20:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202182005-c428410c_37e97751-c768-4104-baa3-699504f452e6 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469215.319188889","description":"Error received from peer ipv4:127.0.0.1:40755","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469215.319175319","description":"Error received from peer ipv4:127.0.0.1:43597","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469215.319175319","description":"Error received from peer ipv4:127.0.0.1:43597","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638469215.319201252","description":"Error received from peer ipv4:127.0.0.1:35481","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 56s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xsgiazmfl4txg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4587
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4587/display/redirect>
Changes:
------------------------------------------
[...truncated 731.05 KB...]
[1m _LOGGER.info(".... Spanner Client created!")[0m
[1m cls._SPANNER_INSTANCE = spanner_client.instance(cls.instance)[0m
[1m> cls._create_database()[0m
[1m[31mapache_beam/io/gcp/experimental/spannerio_read_it_test.py[0m:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mapache_beam/io/gcp/experimental/spannerio_read_it_test.py[0m:78: in _create_database
[1m operation = database.create()[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/spanner_v1/database.py[0m:272: in create
[1m metadata=metadata,[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/spanner_admin_database_v1/gapic/database_admin_client.py[0m:334: in create_database
[1m request, retry=retry, timeout=timeout, metadata=metadata[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py[0m:145: in __call__
[1m return wrapped_func(*args, **kwargs)[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/api_core/retry.py[0m:291: in retry_wrapped_func
[1m on_error=on_error,[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/api_core/retry.py[0m:189: in retry_target
[1m return target()[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/api_core/timeout.py[0m:214: in func_with_timeout
[1m return func(*args, **kwargs)[0m
[1m[31m../../build/gradleenv/1398941891/lib/python3.7/site-packages/google/api_core/grpc_helpers.py[0m:69: in error_remapped_callable
[1m six.raise_from(exceptions.from_grpc_error(exc), exc)[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
value = None
from_value = <_InactiveRpcError of RPC that terminated with:
status = StatusCode.NOT_FOUND
details = "Instance not found: project...le_line":1063,"grpc_message":"Instance not found: projects/apache-beam-testing/instances/beam-test","grpc_status":5}"
>
[1m> ???[0m
[1m[31mE google.api_core.exceptions.NotFound: 404 Instance not found: projects/apache-beam-testing/instances/beam-test[0m
[1m[31m<string>[0m:3: NotFound
------------------------------ Captured log setup ------------------------------
[32mINFO [0m apache_beam.io.gcp.experimental.spannerio_read_it_test:spannerio_read_it_test.py:92 .... PyVersion ---> 3.7.10 (default, Feb 20 2021, 21:21:24)
[GCC 5.4.0 20160609]
[32mINFO [0m apache_beam.io.gcp.experimental.spannerio_read_it_test:spannerio_read_it_test.py:93 .... Setting up!
[32mINFO [0m apache_beam.io.gcp.experimental.spannerio_read_it_test:spannerio_read_it_test.py:102 .... Spanner Client created!
[32mINFO [0m apache_beam.io.gcp.experimental.spannerio_read_it_test:spannerio_read_it_test.py:68 Creating test database: pybeam-read-8f7180664914ffb
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
apache_beam/io/gcp/bigquery.py:2122
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2122: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
apache_beam/io/gcp/bigquery.py:2458
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2458: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project_id = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
apache_beam/io/gcp/bigquery.py:2486
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2486: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanupTempDatasets(project_to_cleanup_pcoll))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/datastore/_gapic.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/datastore/_gapic.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/datastore/_gapic.py>:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/1398941891/lib/python3.7/site-packages/google/cloud/datastore/_gapic.py>:42: PendingDeprecationWarning: The `channel` argument is deprecated; use `transport` instead.
channel=channel, client_info=client._client_info
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-direct-py37.xml> -
[31m[1m======== 29 passed, 1 skipped, 109 warnings, 15 error in 123.76 seconds ========[0m
> Task :sdks:python:test-suites:direct:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:io:google-cloud-platform:jar'.
> java.lang.NullPointerException (no error message)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 22m 17s
199 actionable tasks: 144 executed, 51 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xgaqnxteddcpo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4586
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4586/display/redirect>
Changes:
------------------------------------------
[...truncated 41.00 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 312812328
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 325391292
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 325525999
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 325592756
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 325654745
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 326057195
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638426240
nanos: 326152563
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:156"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.222814321517944 seconds.
INFO:root:Successfully completed job in 7.222814321517944 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36237
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0c88c6b290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0c88c6b320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0c88c6ba70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-templv_pv9t0/artifactstva965h5' '--job-port' '36055' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38217'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41689'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36055'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_34750084-c4c6-474d-ab33-d8e27fb3602f.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_34750084-c4c6-474d-ab33-d8e27fb3602f.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_34750084-c4c6-474d-ab33-d8e27fb3602f.null.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_34750084-c4c6-474d-ab33-d8e27fb3602f.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1202062405-ec39bdcf_ac2a2178-218c-411e-96f5-82040dd62fe9'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:05 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1202062405-ec39bdcf_ac2a2178-218c-411e-96f5-82040dd62fe9'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:06 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:06 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:07 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:07 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1202062405-ec39bdcf_ac2a2178-218c-411e-96f5-82040dd62fe9 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42873.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45099.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36847
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202062405-ec39bdcf_ac2a2178-218c-411e-96f5-82040dd62fe9: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:08 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/12/02 06:24:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1202062405-ec39bdcf_ac2a2178-218c-411e-96f5-82040dd62fe9 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638426249.634093707","description":"Error received from peer ipv4:127.0.0.1:36847","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638426249.634110175","description":"Error received from peer ipv4:127.0.0.1:45099","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638426249.634128470","description":"Error received from peer ipv4:127.0.0.1:42873","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638426249.634093707","description":"Error received from peer ipv4:127.0.0.1:36847","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 1s
214 actionable tasks: 165 executed, 45 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wrmyu3urdwr7g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4585 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4585 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4585/ to view the results.
beam_PostCommit_Python37 - Build # 4584 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4584 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4584/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4583
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4583/display/redirect>
Changes:
------------------------------------------
[...truncated 48.39 MB...]
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
apache_beam/io/gcp/bigquery.py:2128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
apache_beam/io/gcp/bigquery.py:2414
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2414: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
apache_beam/io/gcp/bigquery.py:2416
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2416: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
apache_beam/io/gcp/bigquery.py:2447
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2447: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
apache_beam/io/gcp/bigquery.py:2118
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2118: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2588
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2588: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2589
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2589: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2602
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2602: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 60 passed, 21 skipped, 178 warnings in 5722.53 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 10s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/j6r3l5gzg3one
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4582 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4582 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4582/ to view the results.
beam_PostCommit_Python37 - Build # 4581 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4581 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4581/ to view the results.
beam_PostCommit_Python37 - Build # 4580 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4580 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4580/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4579
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4579/display/redirect?page=changes>
Changes:
[melissapa] [BEAM-11758] Update basics page: Trigger, State and timers
------------------------------------------
[...truncated 48.47 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 60 passed, 21 skipped, 178 warnings in 5779.25 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 25s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wk4khttys4wsu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4578
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4578/display/redirect>
Changes:
------------------------------------------
[...truncated 48.40 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
apache_beam/dataframe/io.py:593
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:593: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 3 failed, 60 passed, 21 skipped, 174 warnings in 6625.85 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 52s
214 actionable tasks: 161 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/hrhoxpvm4jaaa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4577 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4577 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4577/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4576
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4576/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16050 from [BEAM-13307][Playground] Support Python
[noreply] Merge pull request #16071 from [BEAM-13306][Playground] Add using of
[noreply] [BEAM-11943] Fix wrong Klio logo being displayed (#16078)
------------------------------------------
[...truncated 47.91 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 180 warnings in 5924.25 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 58s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7n5gwgp4mu7d6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4575 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4575 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4575/ to view the results.
beam_PostCommit_Python37 - Build # 4574 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4574 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4574/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4573
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4573/display/redirect>
Changes:
------------------------------------------
[...truncated 36.65 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 319183111
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 439570426
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 464204311
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 464384794
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 464474916
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 464544296
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 465000152
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638145216
nanos: 465125322
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.546860218048096 seconds.
INFO:root:Successfully completed job in 8.546860218048096 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:32933
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7feb3625c290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7feb3625c320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7feb3625ca70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp55l2hryb/artifacts9uhgtj90' '--job-port' '37719' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40869'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34315'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:37719'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9a2c4b0e-75e6-4013-92b5-841c81f191ac.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9a2c4b0e-75e6-4013-92b5-841c81f191ac.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9a2c4b0e-75e6-4013-92b5-841c81f191ac.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:21 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9a2c4b0e-75e6-4013-92b5-841c81f191ac.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1129002022-a8ddce42_48a568cb-cee8-4556-8fc3-b8e03e14e660'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:22 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1129002022-a8ddce42_48a568cb-cee8-4556-8fc3-b8e03e14e660'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:22 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:22 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:23 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:23 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:23 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1129002022-a8ddce42_48a568cb-cee8-4556-8fc3-b8e03e14e660 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44719.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37835.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38061
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129002022-a8ddce42_48a568cb-cee8-4556-8fc3-b8e03e14e660: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/29 00:20:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1129002022-a8ddce42_48a568cb-cee8-4556-8fc3-b8e03e14e660 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145226.451676823","description":"Error received from peer ipv4:127.0.0.1:38061","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145226.451696758","description":"Error received from peer ipv4:127.0.0.1:37835","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145226.451705656","description":"Error received from peer ipv4:127.0.0.1:44719","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638145226.451676823","description":"Error received from peer ipv4:127.0.0.1:38061","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 35s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/tyvahgimtizpg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4572
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4572/display/redirect>
Changes:
------------------------------------------
[...truncated 45.46 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 609365940
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 645705223
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 645883083
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 645954608
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 646020650
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 646981477
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638123610
nanos: 647119283
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.27771782875061 seconds.
INFO:root:Successfully completed job in 7.27771782875061 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34379
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f596580e290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f596580e320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f596580ea70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempd09rgrpd/artifactsfr31ip3k' '--job-port' '33837' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37321'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45847'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33837'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_5249b9c1-ab6f-47c4-a63a-c7efde00bdbb.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_5249b9c1-ab6f-47c4-a63a-c7efde00bdbb.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_5249b9c1-ab6f-47c4-a63a-c7efde00bdbb.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_5249b9c1-ab6f-47c4-a63a-c7efde00bdbb.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1128182017-b72705_5ed54b4c-9b89-4005-85a5-f6acfb80b7ac'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1128182017-b72705_5ed54b4c-9b89-4005-85a5-f6acfb80b7ac'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:17 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:19 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1128182017-b72705_5ed54b4c-9b89-4005-85a5-f6acfb80b7ac on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42911.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41065.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44073
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128182017-b72705_5ed54b4c-9b89-4005-85a5-f6acfb80b7ac: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 18:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128182017-b72705_5ed54b4c-9b89-4005-85a5-f6acfb80b7ac finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638123622.189192833","description":"Error received from peer ipv4:127.0.0.1:44073","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638123622.189192833","description":"Error received from peer ipv4:127.0.0.1:44073","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638123622.189202941","description":"Error received from peer ipv4:127.0.0.1:41065","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638123622.189202956","description":"Error received from peer ipv4:127.0.0.1:42911","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 17s
214 actionable tasks: 151 executed, 59 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4enxbngs5ucnc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4571
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4571/display/redirect>
Changes:
------------------------------------------
[...truncated 32.41 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638101960
nanos: 971715927
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 76196432
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91022014
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91203927
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91281652
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91347932
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91825246
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638101961
nanos: 91948747
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 4.992279767990112 seconds.
INFO:root:Successfully completed job in 4.992279767990112 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35275
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc4e2387290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fc4e2387320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc4e2387a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp2ajksjw1/artifactsut6zuisk' '--job-port' '32909' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41337'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42975'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:32909'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:25 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bd901e51-234e-40fb-8a9e-09c233a7af94.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bd901e51-234e-40fb-8a9e-09c233a7af94.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_bd901e51-234e-40fb-8a9e-09c233a7af94.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bd901e51-234e-40fb-8a9e-09c233a7af94.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1128121926-1c615946_81221c64-1f8b-4e74-aaaf-db28a2022eae'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:27 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1128121926-1c615946_81221c64-1f8b-4e74-aaaf-db28a2022eae'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:27 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:29 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:29 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1128121926-1c615946_81221c64-1f8b-4e74-aaaf-db28a2022eae on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39381.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39771.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41311
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:31 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128121926-1c615946_81221c64-1f8b-4e74-aaaf-db28a2022eae: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 12:19:32 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128121926-1c615946_81221c64-1f8b-4e74-aaaf-db28a2022eae finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638101973.396473004","description":"Error received from peer ipv4:127.0.0.1:41311","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638101973.396500848","description":"Error received from peer ipv4:127.0.0.1:39771","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638101973.396473004","description":"Error received from peer ipv4:127.0.0.1:41311","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1638101973.396519261","description":"Error received from peer ipv4:127.0.0.1:39381","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 33s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/d6gqwlkez3cx2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4570 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4570 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4570/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4569
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4569/display/redirect>
Changes:
------------------------------------------
[...truncated 38.68 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 133773326
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 238810539
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 253201007
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 253388643
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 253488779
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 253612041
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 254668712
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638058827
nanos: 254815340
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.530155181884766 seconds.
INFO:root:Successfully completed job in 5.530155181884766 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34207
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd52a018290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd52a018320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd52a018a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempr52u07p6/artifactsf6hk2oau' '--job-port' '58807' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40203'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34159'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:58807'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:31 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1aeb97b5-66cc-4dcc-a2c0-31a3eede5800.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1aeb97b5-66cc-4dcc-a2c0-31a3eede5800.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1aeb97b5-66cc-4dcc-a2c0-31a3eede5800.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1aeb97b5-66cc-4dcc-a2c0-31a3eede5800.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1128002032-8318af13_a48b04f9-efa2-49a9-a1f0-cdb6ab67c671'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:32 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1128002032-8318af13_a48b04f9-efa2-49a9-a1f0-cdb6ab67c671'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:33 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:33 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:34 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:34 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1128002032-8318af13_a48b04f9-efa2-49a9-a1f0-cdb6ab67c671 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42203.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38757.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44469
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128002032-8318af13_a48b04f9-efa2-49a9-a1f0-cdb6ab67c671: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/28 00:20:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1128002032-8318af13_a48b04f9-efa2-49a9-a1f0-cdb6ab67c671 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058837.031887994","description":"Error received from peer ipv4:127.0.0.1:38757","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058837.031876016","description":"Error received from peer ipv4:127.0.0.1:44469","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058837.031900919","description":"Error received from peer ipv4:127.0.0.1:42203","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638058837.031876016","description":"Error received from peer ipv4:127.0.0.1:44469","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 43m 0s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/za5zwyks5qnr2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4568
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4568/display/redirect>
Changes:
------------------------------------------
[...truncated 707.27 KB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 86776733
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 231813430
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 262404441
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 262655973
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 262750148
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 262826442
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 263300657
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1638037305
nanos: 263427495
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.496490716934204 seconds.
INFO:root:Successfully completed job in 7.496490716934204 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40063
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f75005e4290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f75005e4320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f75005e4a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp7cjztp6i/artifactsenjwp946' '--job-port' '56271' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:56271.
WARNING:root:Waiting for grpc channel to be ready at localhost:56271.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43957'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33817'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56271'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:56271.
WARNING:root:Waiting for grpc channel to be ready at localhost:56271.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:58 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a46e6287-cae2-41e9-b23b-6a004e98ba57.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:58 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a46e6287-cae2-41e9-b23b-6a004e98ba57.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:58 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a46e6287-cae2-41e9-b23b-6a004e98ba57.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:58 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a46e6287-cae2-41e9-b23b-6a004e98ba57.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:59 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1127182159-c03b6d3e_a98d4b9c-bbec-41ee-a211-6c9a78c116e3'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:21:59 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1127182159-c03b6d3e_a98d4b9c-bbec-41ee-a211-6c9a78c116e3'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:01 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:02 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/11/27 18:22:09 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:10 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:10 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1127182159-c03b6d3e_a98d4b9c-bbec-41ee-a211-6c9a78c116e3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43519.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45865.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:38023
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:12 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127182159-c03b6d3e_a98d4b9c-bbec-41ee-a211-6c9a78c116e3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.11 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 18:22:13 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127182159-c03b6d3e_a98d4b9c-bbec-41ee-a211-6c9a78c116e3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638037334.606391720","description":"Error received from peer ipv4:127.0.0.1:38023","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638037334.606418407","description":"Error received from peer ipv4:127.0.0.1:43519","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638037334.606418436","description":"Error received from peer ipv4:127.0.0.1:45865","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1638037334.606391720","description":"Error received from peer ipv4:127.0.0.1:38023","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 13s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2r5s5a5oi25lm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4567
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4567/display/redirect>
Changes:
------------------------------------------
[...truncated 48.02 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 5664.25 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 3s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jv5f2yzjc5j3u
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4566
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4566/display/redirect>
Changes:
------------------------------------------
[...truncated 46.54 MB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 28711318
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 28892517
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 28977632
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 29046058
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 30359983
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637993986
nanos: 30504703
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.6554319858551025 seconds.
INFO:root:Successfully completed job in 5.6554319858551025 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37563
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9e15162290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9e15162320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f9e15162a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempdu10gagv/artifactss7hm6l3h' '--job-port' '46919' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39125'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33755'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:46919'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_53fb6fc8-2198-48d2-ba1d-9bdbfbc2e5ae.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_53fb6fc8-2198-48d2-ba1d-9bdbfbc2e5ae.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_53fb6fc8-2198-48d2-ba1d-9bdbfbc2e5ae.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_53fb6fc8-2198-48d2-ba1d-9bdbfbc2e5ae.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:51 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1127061951-758559fc_e54176bd-f87a-47b6-a848-c3e050483138'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:51 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1127061951-758559fc_e54176bd-f87a-47b6-a848-c3e050483138'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:51 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:51 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:52 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:52 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1127061951-758559fc_e54176bd-f87a-47b6-a848-c3e050483138 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35481.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:32995.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43359
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127061951-758559fc_e54176bd-f87a-47b6-a848-c3e050483138: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 06:19:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127061951-758559fc_e54176bd-f87a-47b6-a848-c3e050483138 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993995.698109687","description":"Error received from peer ipv4:127.0.0.1:43359","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993995.698109687","description":"Error received from peer ipv4:127.0.0.1:43359","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993995.698138662","description":"Error received from peer ipv4:127.0.0.1:35481","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637993995.698122597","description":"Error received from peer ipv4:127.0.0.1:32995","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 41s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4uyrgx2fog4wg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4565
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4565/display/redirect>
Changes:
------------------------------------------
[...truncated 147.33 KB...]
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 725639104
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 843518018
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 871046304
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 871218681
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 871299743
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 871451854
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 873005390
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637972647
nanos: 873143911
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.026279926300049 seconds.
INFO:root:Successfully completed job in 7.026279926300049 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34233
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f07f8683290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f07f8683320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f07f8683a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempycsd4xvq/artifacts83so25tg' '--job-port' '38075' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:38075.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46409'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37613'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:38075'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:15 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:38075.
WARNING:root:Waiting for grpc channel to be ready at localhost:38075.
WARNING:root:Waiting for grpc channel to be ready at localhost:38075.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_98a857fc-4214-4a61-9d00-c20ae0fb1006.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_98a857fc-4214-4a61-9d00-c20ae0fb1006.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_98a857fc-4214-4a61-9d00-c20ae0fb1006.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:20 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_98a857fc-4214-4a61-9d00-c20ae0fb1006.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:20 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1127002420-6da623f5_d9f0a33f-89a4-4227-af9e-57bbb758611d'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:21 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1127002420-6da623f5_d9f0a33f-89a4-4227-af9e-57bbb758611d'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:21 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:22 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b"21/11/27 00:24:25 WARN org.apache.spark.util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041."
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:25 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:25 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:25 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1127002420-6da623f5_d9f0a33f-89a4-4227-af9e-57bbb758611d on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38841.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46201.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33887
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:27 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127002420-6da623f5_d9f0a33f-89a4-4227-af9e-57bbb758611d: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/27 00:24:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1127002420-6da623f5_d9f0a33f-89a4-4227-af9e-57bbb758611d finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637972669.838158296","description":"Error received from peer ipv4:127.0.0.1:38841","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637972669.838152408","description":"Error received from peer ipv4:127.0.0.1:33887","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637972669.838152408","description":"Error received from peer ipv4:127.0.0.1:33887","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637972669.838152409","description":"Error received from peer ipv4:127.0.0.1:46201","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 44m 38s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ko77e6az2h4ve
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4564/display/redirect>
Changes:
------------------------------------------
[...truncated 29.98 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 19870281
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 175257682
}
message: "Renamed 1 shards in 0.16 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 194816112
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 195003032
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 195079088
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 195146322
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 196402549
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637951152
nanos: 196558952
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.705299377441406 seconds.
INFO:root:Successfully completed job in 7.705299377441406 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:38985
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f19edc16290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f19edc16320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f19edc16a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempr467u9yr/artifactsf_xbjz_i' '--job-port' '40483' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46113'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34689'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40483'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:56 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_bfd9262b-c641-491a-97f8-77ddbd855cf4.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_bfd9262b-c641-491a-97f8-77ddbd855cf4.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_bfd9262b-c641-491a-97f8-77ddbd855cf4.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_bfd9262b-c641-491a-97f8-77ddbd855cf4.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:58 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1126182558-37cc7a79_c2c9dcf0-a286-44e8-bc53-95fa659d11e9'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:58 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1126182558-37cc7a79_c2c9dcf0-a286-44e8-bc53-95fa659d11e9'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:58 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:58 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:59 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:59 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:25:59 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1126182558-37cc7a79_c2c9dcf0-a286-44e8-bc53-95fa659d11e9 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35127.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45631.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35335
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:01 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126182558-37cc7a79_c2c9dcf0-a286-44e8-bc53-95fa659d11e9: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:02 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/26 18:26:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1126182558-37cc7a79_c2c9dcf0-a286-44e8-bc53-95fa659d11e9 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637951162.765638943","description":"Error received from peer ipv4:127.0.0.1:35335","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637951162.765638943","description":"Error received from peer ipv4:127.0.0.1:35335","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637951162.765691921","description":"Error received from peer ipv4:127.0.0.1:35127","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637951162.765644697","description":"Error received from peer ipv4:127.0.0.1:45631","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 15s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/yfsxjgbzdbgow
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4563/display/redirect>
Changes:
------------------------------------------
[...truncated 51.22 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 6759.90 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 59m 43s
214 actionable tasks: 156 executed, 54 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mhqbcnqzw7erc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4562 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4562 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4562/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4561/display/redirect>
Changes:
------------------------------------------
[...truncated 48.12 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 180 warnings in 5833.99 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 2s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/p675ycgzzw3ne
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4560/display/redirect>
Changes:
------------------------------------------
[...truncated 48.02 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6700.71 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 12s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/huc7n3lyy5o5s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4559/display/redirect>
Changes:
------------------------------------------
[...truncated 30.15 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 191779375
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 315219640
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 336171627
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 336339712
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 336412668
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 336487770
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 337394237
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637842841
nanos: 337525606
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.171581745147705 seconds.
INFO:root:Successfully completed job in 10.171581745147705 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46179
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f90e3a96290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f90e3a96320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f90e3a96a70> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzq5fggpp/artifactsf2psj99v' '--job-port' '33179' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:33179.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35991'
WARNING:root:Waiting for grpc channel to be ready at localhost:33179.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38025'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:33179'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:33179.
WARNING:root:Waiting for grpc channel to be ready at localhost:33179.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_485bc434-08b8-4bfe-a434-6bf5acd3eaed.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_485bc434-08b8-4bfe-a434-6bf5acd3eaed.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_485bc434-08b8-4bfe-a434-6bf5acd3eaed.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_485bc434-08b8-4bfe-a434-6bf5acd3eaed.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1125122055-b65790c0_2416d465-e513-472e-913d-6a0c35286a61'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:55 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1125122055-b65790c0_2416d465-e513-472e-913d-6a0c35286a61'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:20:58 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:01 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:02 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1125122055-b65790c0_2416d465-e513-472e-913d-6a0c35286a61 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33841.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40521.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46281
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1125122055-b65790c0_2416d465-e513-472e-913d-6a0c35286a61: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/25 12:21:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1125122055-b65790c0_2416d465-e513-472e-913d-6a0c35286a61 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637842866.609330111","description":"Error received from peer ipv4:127.0.0.1:46281","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637842866.609341739","description":"Error received from peer ipv4:127.0.0.1:40521","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637842866.609330111","description":"Error received from peer ipv4:127.0.0.1:46281","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637842866.609360670","description":"Error received from peer ipv4:127.0.0.1:33841","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 36s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7faa56ixyhxnc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4558/display/redirect>
Changes:
------------------------------------------
[...truncated 46.31 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6772.99 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 1s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/xegff7vsbev4c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4557 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4557 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4557/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4556/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Simplify
------------------------------------------
[...truncated 4.01 MB...]
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 752172231
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 888167858
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 896175146
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 896359443
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 896476984
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 896595954
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 897679805
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637777928
nanos: 897789716
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.839424133300781 seconds.
INFO:root:Successfully completed job in 8.839424133300781 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35501
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ffba2bc2e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ffba2bc2ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ffba2bc5680> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempdfy8k9fz/artifactsp502h2v3' '--job-port' '57517' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:57517.
WARNING:root:Waiting for grpc channel to be ready at localhost:57517.
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:18:57 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38631'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:18:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45695'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:18:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:57517'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:18:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:57517.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_ef83b4e3-c361-41cf-913f-712f17da057d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_ef83b4e3-c361-41cf-913f-712f17da057d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_ef83b4e3-c361-41cf-913f-712f17da057d.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:00 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_ef83b4e3-c361-41cf-913f-712f17da057d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:01 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1124181901-8e2f2568_637932fe-8203-42da-a003-51a6efa945a7'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:01 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1124181901-8e2f2568_637932fe-8203-42da-a003-51a6efa945a7'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:02 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:07 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:07 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:07 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1124181901-8e2f2568_637932fe-8203-42da-a003-51a6efa945a7 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41805.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42539.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35125
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:09 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1124181901-8e2f2568_637932fe-8203-42da-a003-51a6efa945a7: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/24 18:19:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1124181901-8e2f2568_637932fe-8203-42da-a003-51a6efa945a7 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637777951.501499264","description":"Error received from peer ipv4:127.0.0.1:35125","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637777951.501518887","description":"Error received from peer ipv4:127.0.0.1:41805","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637777951.501499264","description":"Error received from peer ipv4:127.0.0.1:35125","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637777951.501518899","description":"Error received from peer ipv4:127.0.0.1:42539","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 23s
214 actionable tasks: 157 executed, 53 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pudgskffrtuvu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4555/display/redirect>
Changes:
------------------------------------------
[...truncated 48.15 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 176 warnings in 6668.88 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 41s
214 actionable tasks: 153 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/orqhsonebowv4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4554/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12829] Never copy .gogradle/** for Python. (#16059)
------------------------------------------
[...truncated 1.27 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 14802217
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 164531230
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 190372705
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 190563678
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 190640449
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 190710783
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 191998243
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637734757
nanos: 192115783
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.960915565490723 seconds.
INFO:root:Successfully completed job in 5.960915565490723 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44671
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f84fb187e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f84fb187ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f84fb189680> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzveej87h/artifactsje16893i' '--job-port' '34973' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41545'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40631'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34973'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e68019d4-2f34-47b8-bfa2-1fd6e7b3eaf4.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e68019d4-2f34-47b8-bfa2-1fd6e7b3eaf4.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e68019d4-2f34-47b8-bfa2-1fd6e7b3eaf4.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:22 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e68019d4-2f34-47b8-bfa2-1fd6e7b3eaf4.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:22 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1124061922-2f135416_118547d6-239b-4de9-aff5-eb9a5a68eba2'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:23 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1124061922-2f135416_118547d6-239b-4de9-aff5-eb9a5a68eba2'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:23 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:23 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:24 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:24 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:24 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1124061922-2f135416_118547d6-239b-4de9-aff5-eb9a5a68eba2 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45165.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46043.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34875
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1124061922-2f135416_118547d6-239b-4de9-aff5-eb9a5a68eba2: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/24 06:19:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1124061922-2f135416_118547d6-239b-4de9-aff5-eb9a5a68eba2 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637734767.293981360","description":"Error received from peer ipv4:127.0.0.1:34875","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637734767.293981360","description":"Error received from peer ipv4:127.0.0.1:34875","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637734767.294001393","description":"Error received from peer ipv4:127.0.0.1:45165","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637734767.293983708","description":"Error received from peer ipv4:127.0.0.1:46043","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 41s
214 actionable tasks: 177 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/cygzbc2pzrdxy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4553 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4553 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4553/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4552/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #16002 from [BEAM-13101] [Playground] load examples
[noreply] Merge pull request #16016 from [BEAM-13089][Playground] Enable
[noreply] Merge pull request #16046 from [BEAM-13295] [Playground] [Bugfix]
[noreply] Merge pull request #16048 from [Beam-13256] [Playground] Implement an
[noreply] Merge pull request #16026 from [BEAM-13208][Playground] Separate stderr
[noreply] Append to state in chunks. (#15983)
------------------------------------------
[...truncated 26.97 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 153776645
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 258419275
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 272938489
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 273170709
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 273260354
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 273329973
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 274814128
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637691485
nanos: 274961709
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.411851406097412 seconds.
INFO:root:Successfully completed job in 6.411851406097412 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36687
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe684f47e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe684f47ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe684f49680> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzi7hux5n/artifacts42df3vr3' '--job-port' '54761' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33485'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41205'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:54761'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:09 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_26eb141c-34c8-40cf-aa8b-5c0541bb330a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_26eb141c-34c8-40cf-aa8b-5c0541bb330a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_26eb141c-34c8-40cf-aa8b-5c0541bb330a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:10 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_26eb141c-34c8-40cf-aa8b-5c0541bb330a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:11 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1123181811-3af5a1e1_67f2e5d0-b872-4411-b2a0-60568137a11a'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:11 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1123181811-3af5a1e1_67f2e5d0-b872-4411-b2a0-60568137a11a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:11 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:11 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:12 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:12 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1123181811-3af5a1e1_67f2e5d0-b872-4411-b2a0-60568137a11a on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39737.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40499.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43151
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:13 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:13 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123181811-3af5a1e1_67f2e5d0-b872-4411-b2a0-60568137a11a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 18:18:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123181811-3af5a1e1_67f2e5d0-b872-4411-b2a0-60568137a11a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637691495.210173687","description":"Error received from peer ipv4:127.0.0.1:43151","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637691495.210205644","description":"Error received from peer ipv4:127.0.0.1:39737","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637691495.210178044","description":"Error received from peer ipv4:127.0.0.1:40499","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637691495.210173687","description":"Error received from peer ipv4:127.0.0.1:43151","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 9s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/oyorihgnxp4ju
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4551/display/redirect>
Changes:
------------------------------------------
[...truncated 29.40 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 828919410
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 942709922
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 973275899
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 973512887
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 973627328
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 973704099
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 974357843
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637669949
nanos: 974557638
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.829068183898926 seconds.
INFO:root:Successfully completed job in 8.829068183898926 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46025
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe0f9b3de60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe0f9b3def0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe0f9b3f680> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp_myh1anq/artifactsaono93rz' '--job-port' '56083' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:56083.
WARNING:root:Waiting for grpc channel to be ready at localhost:56083.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:20 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35095'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44343'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56083'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:56083.
WARNING:root:Waiting for grpc channel to be ready at localhost:56083.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9d1d1cfb-e0d2-4a8a-b05a-f5dd2bb20ff5.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9d1d1cfb-e0d2-4a8a-b05a-f5dd2bb20ff5.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9d1d1cfb-e0d2-4a8a-b05a-f5dd2bb20ff5.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:24 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9d1d1cfb-e0d2-4a8a-b05a-f5dd2bb20ff5.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:25 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1123121925-93447203_938860c2-31d1-48b7-b261-7fa90f847fc1'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:25 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1123121925-93447203_938860c2-31d1-48b7-b261-7fa90f847fc1'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:31 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:31 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:31 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1123121925-93447203_938860c2-31d1-48b7-b261-7fa90f847fc1 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41569.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45441.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45619
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:33 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123121925-93447203_938860c2-31d1-48b7-b261-7fa90f847fc1: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 12:19:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123121925-93447203_938860c2-31d1-48b7-b261-7fa90f847fc1 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637669975.158675853","description":"Error received from peer ipv4:127.0.0.1:45619","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637669975.158694494","description":"Error received from peer ipv4:127.0.0.1:41569","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637669975.158675853","description":"Error received from peer ipv4:127.0.0.1:45619","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637669975.158682860","description":"Error received from peer ipv4:127.0.0.1:45441","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 28s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/23utyzxzxsuf4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4550/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-13003] Integration Test for DebeziumIO that connects to
[noreply] Merge pull request #16032 from [BEAM-13297][Playground][Bugfix]
[noreply] Merge pull request #16031 from [BEAM-13296][Playground][Bugfix] remove
[noreply] Merge pull request #16030 from [BEAM-13219][Playground] Initialise
[noreply] Merge pull request #15996 from [BEAM-13156][Playground] Implement
------------------------------------------
[...truncated 48.03 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 180 warnings in 5826.62 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 39s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ofrcadwqg52ro
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4549/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Make sure implicit inputs are noted in the portable graph.
------------------------------------------
[...truncated 312.13 KB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 322662591
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-13"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 432865619
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-13"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 439189434
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 439406156
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 439532995
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 439658164
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 441540956
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637627590
nanos: 441739797
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 11.69642949104309 seconds.
INFO:root:Successfully completed job in 11.69642949104309 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40543
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8598c7fe60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8598c7fef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8598c81680> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempzdfnmfzv/artifactsjrgkhc0r' '--job-port' '35109' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:32769'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36471'
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:35109'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
WARNING:root:Waiting for grpc channel to be ready at localhost:35109.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_87c55ae2-e574-49bc-a993-43bed00f7f99.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_87c55ae2-e574-49bc-a993-43bed00f7f99.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_87c55ae2-e574-49bc-a993-43bed00f7f99.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:30 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_87c55ae2-e574-49bc-a993-43bed00f7f99.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:32 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1123003332-b4275852_9d221255-5d9a-41a1-a91d-a189adb586f3'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:32 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1123003332-b4275852_9d221255-5d9a-41a1-a91d-a189adb586f3'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:33 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:35 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:39 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:39 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1123003332-b4275852_9d221255-5d9a-41a1-a91d-a189adb586f3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42283.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46497.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36469
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123003332-b4275852_9d221255-5d9a-41a1-a91d-a189adb586f3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/23 00:33:45 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1123003332-b4275852_9d221255-5d9a-41a1-a91d-a189adb586f3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637627626.483125289","description":"Error received from peer ipv4:127.0.0.1:36469","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637627626.483125289","description":"Error received from peer ipv4:127.0.0.1:36469","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637627626.486146333","description":"Error received from peer ipv4:127.0.0.1:46497","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637627626.487470337","description":"Error received from peer ipv4:127.0.0.1:42283","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 2s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fs2m7p7mfpiss
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4548/display/redirect>
Changes:
------------------------------------------
[...truncated 43.22 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 752850294
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 897146940
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 929059743
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 929296731
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 929389715
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 929463148
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 930605173
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637605076
nanos: 930759906
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.391949892044067 seconds.
INFO:root:Successfully completed job in 8.391949892044067 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39827
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe890d17dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe890d17e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe890d195f0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempsqxz5ugm/artifactsn4yw4gzh' '--job-port' '34293' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41869'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:33393'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:34293'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_35021b50-f2d9-4346-9b2b-f9a60547eac8.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_35021b50-f2d9-4346-9b2b-f9a60547eac8.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_35021b50-f2d9-4346-9b2b-f9a60547eac8.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_35021b50-f2d9-4346-9b2b-f9a60547eac8.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1122181802-9dac9813_fa267be5-99e1-4ad2-a397-56edc7c3d195'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:03 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1122181802-9dac9813_fa267be5-99e1-4ad2-a397-56edc7c3d195'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1122181802-9dac9813_fa267be5-99e1-4ad2-a397-56edc7c3d195 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39647.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34617.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34423
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1122181802-9dac9813_fa267be5-99e1-4ad2-a397-56edc7c3d195: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/22 18:18:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1122181802-9dac9813_fa267be5-99e1-4ad2-a397-56edc7c3d195 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637605087.332801665","description":"Error received from peer ipv4:127.0.0.1:39647","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637605087.332801528","description":"Error received from peer ipv4:127.0.0.1:34617","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637605087.332776827","description":"Error received from peer ipv4:127.0.0.1:34423","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637605087.332776827","description":"Error received from peer ipv4:127.0.0.1:34423","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 8s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wy4stlgnzkwgw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4547/display/redirect>
Changes:
------------------------------------------
[...truncated 36.38 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 845982313
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 980501413
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 990662813
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 990852832
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 990928649
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 990992307
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 991459608
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637583965
nanos: 991574287
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.320575952529907 seconds.
INFO:root:Successfully completed job in 8.320575952529907 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35947
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f51d4cc0dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f51d4cc0e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f51d4cc25f0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempb_nsdoca/artifacts9vqr2vfc' '--job-port' '50519' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34137'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39275'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50519'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_42283cf9-1123-42fc-af30-a37575f44d46.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_42283cf9-1123-42fc-af30-a37575f44d46.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_42283cf9-1123-42fc-af30-a37575f44d46.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:11 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_42283cf9-1123-42fc-af30-a37575f44d46.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:12 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1122122612-b8b6c5ad_743b1501-5c8f-4dd0-8067-0eaad63ee137'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:12 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1122122612-b8b6c5ad_743b1501-5c8f-4dd0-8067-0eaad63ee137'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:12 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:13 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:14 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1122122612-b8b6c5ad_743b1501-5c8f-4dd0-8067-0eaad63ee137 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41545.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37387.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46101
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1122122612-b8b6c5ad_743b1501-5c8f-4dd0-8067-0eaad63ee137: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/22 12:26:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1122122612-b8b6c5ad_743b1501-5c8f-4dd0-8067-0eaad63ee137 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637583977.319311423","description":"Error received from peer ipv4:127.0.0.1:46101","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637583977.319354377","description":"Error received from peer ipv4:127.0.0.1:41545","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637583977.319311423","description":"Error received from peer ipv4:127.0.0.1:46101","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637583977.319311075","description":"Error received from peer ipv4:127.0.0.1:37387","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 58s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vz2l3oysbtsoe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4546/display/redirect>
Changes:
------------------------------------------
[...truncated 47.79 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6653.36 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 35s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/5byeujuyjekno
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4545 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4545 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4545/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4544
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4544/display/redirect>
Changes:
------------------------------------------
[...truncated 40.99 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 594254255
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 734490633
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 769996881
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 770177602
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 770257234
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 770353555
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 770846366
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637519153
nanos: 770957469
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.365418910980225 seconds.
INFO:root:Successfully completed job in 8.365418910980225 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44227
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f8746682ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f8746682f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f8746685710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempsbuq0_x0/artifactsjgab28u9' '--job-port' '37019' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:37019.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39497'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37643'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:37019'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:37019.
WARNING:root:Waiting for grpc channel to be ready at localhost:37019.
WARNING:root:Waiting for grpc channel to be ready at localhost:37019.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e8f29fdb-3c79-442c-aed9-1a01e05216dc.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e8f29fdb-3c79-442c-aed9-1a01e05216dc.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e8f29fdb-3c79-442c-aed9-1a01e05216dc.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:06 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e8f29fdb-3c79-442c-aed9-1a01e05216dc.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:07 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1121182607-5707f94_9e8ed05b-6b2f-420a-b436-d5b5af0003a1'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:07 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1121182607-5707f94_9e8ed05b-6b2f-420a-b436-d5b5af0003a1'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:08 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:09 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:12 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:12 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1121182607-5707f94_9e8ed05b-6b2f-420a-b436-d5b5af0003a1 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43495.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:14 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33431.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33427
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:14 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:15 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121182607-5707f94_9e8ed05b-6b2f-420a-b436-d5b5af0003a1: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 18:26:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121182607-5707f94_9e8ed05b-6b2f-420a-b436-d5b5af0003a1 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637519176.952925216","description":"Error received from peer ipv4:127.0.0.1:33431","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637519176.952919696","description":"Error received from peer ipv4:127.0.0.1:33427","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637519176.952960874","description":"Error received from peer ipv4:127.0.0.1:43495","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637519176.952919696","description":"Error received from peer ipv4:127.0.0.1:33427","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 19s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ad7e43b2jm4ie
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4543/display/redirect>
Changes:
------------------------------------------
[...truncated 48.09 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 182 warnings in 5736.08 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 43s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kzefakyfw34b6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4542/display/redirect>
Changes:
------------------------------------------
[...truncated 40.82 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 635957241
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 793161153
}
message: "Renamed 1 shards in 0.16 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 811218738
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 811377525
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 811463832
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 811527013
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 812711238
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637475530
nanos: 812821149
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.709824323654175 seconds.
INFO:root:Successfully completed job in 5.709824323654175 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33455
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f9847759ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f9847759f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f984775c710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempv8on2dfp/artifactsi542uo71' '--job-port' '45663' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36493'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42131'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:45663'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_6431c2f7-ff95-4571-bb21-5b0b780dbf4a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_6431c2f7-ff95-4571-bb21-5b0b780dbf4a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_6431c2f7-ff95-4571-bb21-5b0b780dbf4a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_6431c2f7-ff95-4571-bb21-5b0b780dbf4a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:56 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1121061856-28478934_2ab1baa0-c5f9-49b6-81c3-4c31b130aa32'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:56 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1121061856-28478934_2ab1baa0-c5f9-49b6-81c3-4c31b130aa32'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:56 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:56 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:57 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:57 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:57 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1121061856-28478934_2ab1baa0-c5f9-49b6-81c3-4c31b130aa32 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43791.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:58 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:58 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46049.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36617
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:58 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:58 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121061856-28478934_2ab1baa0-c5f9-49b6-81c3-4c31b130aa32: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 06:18:59 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121061856-28478934_2ab1baa0-c5f9-49b6-81c3-4c31b130aa32 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637475540.329867144","description":"Error received from peer ipv4:127.0.0.1:46049","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637475540.329849363","description":"Error received from peer ipv4:127.0.0.1:36617","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637475540.329884211","description":"Error received from peer ipv4:127.0.0.1:43791","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637475540.329849363","description":"Error received from peer ipv4:127.0.0.1:36617","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 2s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/phi6sor7vyq2q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4541
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4541/display/redirect>
Changes:
------------------------------------------
[...truncated 41.04 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 401126146
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 572100400
}
message: "Renamed 1 shards in 0.17 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 576640367
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 576802492
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 576937198
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 577047824
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 577522277
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637454035
nanos: 577627897
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.857519865036011 seconds.
INFO:root:Successfully completed job in 5.857519865036011 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39515
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3d5924aef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3d5924af80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3d5924d710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempd8zd61ou/artifactsxev_0jj7' '--job-port' '60875' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:60875.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:35491'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40421'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:60875'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:60875.
WARNING:root:Waiting for grpc channel to be ready at localhost:60875.
WARNING:root:Waiting for grpc channel to be ready at localhost:60875.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_2d3d3bc5-b7e4-4959-8095-24407c4d0d8d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_2d3d3bc5-b7e4-4959-8095-24407c4d0d8d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:47 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_2d3d3bc5-b7e4-4959-8095-24407c4d0d8d.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_2d3d3bc5-b7e4-4959-8095-24407c4d0d8d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:48 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1121002048-471cc6bc_9e588d0f-f795-4791-936e-d2255726bd08'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:48 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1121002048-471cc6bc_9e588d0f-f795-4791-936e-d2255726bd08'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:49 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:50 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:53 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:53 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1121002048-471cc6bc_9e588d0f-f795-4791-936e-d2255726bd08 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36133.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:39365.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34981
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121002048-471cc6bc_9e588d0f-f795-4791-936e-d2255726bd08: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/21 00:20:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1121002048-471cc6bc_9e588d0f-f795-4791-936e-d2255726bd08 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637454057.265802728","description":"Error received from peer ipv4:127.0.0.1:34981","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1637454057.265876061","description":"Error received from peer ipv4:127.0.0.1:36133","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637454057.265802728","description":"Error received from peer ipv4:127.0.0.1:34981","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637454057.265852327","description":"Error received from peer ipv4:127.0.0.1:39365","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 17s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/dtvdn67vyi5q6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4540
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4540/display/redirect>
Changes:
------------------------------------------
[...truncated 39.85 MB...]
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 305429935
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 321170091
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 321409702
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 321547746
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 321678400
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 322319984
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637432833
nanos: 322476625
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.056329727172852 seconds.
INFO:root:Successfully completed job in 8.056329727172852 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36023
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f2f12a36ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f2f12a36f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f2f12a39710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp826r71vy/artifactsyu3alxjl' '--job-port' '41835' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:41835.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42273'
WARNING:root:Waiting for grpc channel to be ready at localhost:41835.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39143'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:41835'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:23 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:41835.
WARNING:root:Waiting for grpc channel to be ready at localhost:41835.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e2c218a6-6baa-4cee-9f9d-3362fb07a916.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e2c218a6-6baa-4cee-9f9d-3362fb07a916.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:26 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e2c218a6-6baa-4cee-9f9d-3362fb07a916.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e2c218a6-6baa-4cee-9f9d-3362fb07a916.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1120182727-79e5ff58_1b2fb2e4-d584-4e1a-83b8-1597977f04ea'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:28 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1120182727-79e5ff58_1b2fb2e4-d584-4e1a-83b8-1597977f04ea'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:29 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:30 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:34 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:34 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:34 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1120182727-79e5ff58_1b2fb2e4-d584-4e1a-83b8-1597977f04ea on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42365.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36303.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35929
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1120182727-79e5ff58_1b2fb2e4-d584-4e1a-83b8-1597977f04ea: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 18:27:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1120182727-79e5ff58_1b2fb2e4-d584-4e1a-83b8-1597977f04ea finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637432861.360221146","description":"Error received from peer ipv4:127.0.0.1:35929","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637432861.360221146","description":"Error received from peer ipv4:127.0.0.1:35929","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637432861.360247975","description":"Error received from peer ipv4:127.0.0.1:42365","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637432861.360221150","description":"Error received from peer ipv4:127.0.0.1:36303","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 31s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fhg3q2vrve22m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4539
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4539/display/redirect>
Changes:
------------------------------------------
[...truncated 24.98 MB...]
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637412321
nanos: 847408056
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 10893106
}
message: "Renamed 1 shards in 0.16 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23005485
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23197889
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23280620
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23345947
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23872137
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637412322
nanos: 23999214
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 28.341896295547485 seconds.
INFO:root:Successfully completed job in 28.341896295547485 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35689
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd21ed04ef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd21ed04f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd21ed07710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempm641tyrn/artifactsfxg150ah' '--job-port' '39753' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:39753.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:41757'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35977'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39753'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:30 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_9a37929b-9180-4e46-8b91-226eba2a9fc1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_9a37929b-9180-4e46-8b91-226eba2a9fc1.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_9a37929b-9180-4e46-8b91-226eba2a9fc1.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:31 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_9a37929b-9180-4e46-8b91-226eba2a9fc1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:31 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1120124531-2d97dd76_05ad9dff-26c8-4518-9581-a94c21ecfa7e'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:32 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1120124531-2d97dd76_05ad9dff-26c8-4518-9581-a94c21ecfa7e'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:32 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:34 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:36 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:36 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1120124531-2d97dd76_05ad9dff-26c8-4518-9581-a94c21ecfa7e on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41303.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37021.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46359
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:38 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1120124531-2d97dd76_05ad9dff-26c8-4518-9581-a94c21ecfa7e: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.14 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/20 12:45:39 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1120124531-2d97dd76_05ad9dff-26c8-4518-9581-a94c21ecfa7e finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637412340.262676019","description":"Error received from peer ipv4:127.0.0.1:46359","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637412340.262687641","description":"Error received from peer ipv4:127.0.0.1:41303","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637412340.262676019","description":"Error received from peer ipv4:127.0.0.1:46359","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637412340.262676920","description":"Error received from peer ipv4:127.0.0.1:37021","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 33s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2akahd4sil6oe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4538
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4538/display/redirect?page=changes>
Changes:
[ningkang0957] Removed Pillow from requirements of containers.
[noreply] Merge pull request #16013 from [BEAM-13281] [playground] Update favicon
[noreply] Merge pull request #15768 from [BEAM-13064] [Playground] Implement
[noreply] Added go code coverage report with codecov (#16020)
------------------------------------------
[...truncated 36.73 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 182 warnings in 5961.42 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 47m 49s
214 actionable tasks: 159 executed, 51 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/zghdesuxennf2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4537
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4537/display/redirect?page=changes>
Changes:
[Valentyn Tymofieiev] Add future back to Beam containers.
[noreply] Merge pull request #16025 from [BEAM-13216] [Playground] Add client to
[ningkang0957] Updated dep and goldens for Interactive Beam screen diff tests
------------------------------------------
[...truncated 50.94 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6791.90 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 59m 2s
214 actionable tasks: 161 executed, 49 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/wfp3a7ixdoxjs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4536 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4536 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4536/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4535
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4535/display/redirect>
Changes:
------------------------------------------
[...truncated 658.92 KB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 655656099
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 797549724
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 830578804
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 830756425
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 830831527
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 830903053
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 831369161
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637324411
nanos: 831484317
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.917880296707153 seconds.
INFO:root:Successfully completed job in 5.917880296707153 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43605
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.36.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f978dedaef0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f978dedaf80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f978dedd710> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.36.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempee_by40l/artifactsv3vrl07j' '--job-port' '58675' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42695'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:46139'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:58675'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:16 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_0da930ea-9524-420a-a2d6-9faf2e31d662.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_0da930ea-9524-420a-a2d6-9faf2e31d662.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_0da930ea-9524-420a-a2d6-9faf2e31d662.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_0da930ea-9524-420a-a2d6-9faf2e31d662.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1119122017-b452a25_bae356c2-1faa-42bb-9f44-b839d400831b'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1119122017-b452a25_bae356c2-1faa-42bb-9f44-b839d400831b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:19 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1119122017-b452a25_bae356c2-1faa-42bb-9f44-b839d400831b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36417.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38565.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36999
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1119122017-b452a25_bae356c2-1faa-42bb-9f44-b839d400831b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/19 12:20:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1119122017-b452a25_bae356c2-1faa-42bb-9f44-b839d400831b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637324421.692143375","description":"Error received from peer ipv4:127.0.0.1:38565","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637324421.692188230","description":"Error received from peer ipv4:127.0.0.1:36417","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637324421.692135103","description":"Error received from peer ipv4:127.0.0.1:36999","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637324421.692135103","description":"Error received from peer ipv4:127.0.0.1:36999","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 20s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2n3ev5lmke6u6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4534 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4534 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4534/ to view the results.
beam_PostCommit_Python37 - Build # 4533 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4533 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4533/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4532
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4532/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #15785 from Support for passing priority parameter in
------------------------------------------
[...truncated 47.90 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6716.34 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 57m 18s
214 actionable tasks: 155 executed, 55 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/oushjzbvjf6g2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4531 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4531 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4531/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4530
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4530/display/redirect>
Changes:
------------------------------------------
[...truncated 48.01 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 5807.54 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 30s
214 actionable tasks: 167 executed, 43 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rupfkrajhfpkg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4529 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4529 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4529/ to view the results.
beam_PostCommit_Python37 - Build # 4528 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4528 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4528/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4527
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4527/display/redirect>
Changes:
------------------------------------------
[...truncated 40.98 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 5920.30 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 11s
214 actionable tasks: 156 executed, 54 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mqguu4er3pw3s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4526
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4526/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #15997: [BEAM-8688] Upgrading gcsio library to latest
------------------------------------------
[...truncated 48.02 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 5692.14 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 55s
214 actionable tasks: 174 executed, 36 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/erx57enalbivk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4525
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4525/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13178][Playground]
[Pablo Estrada] [BEAM-8033] Throwing an error if a dataSourceProviderFn is defined twice
[minherz] fix: update dlp dependency version range
[noreply] Merge pull request #15740 from [BEAM-12936][Playground] Code editor -
[noreply] Merge pull request #15976 from [Playground][BEAM-12941][Bugfix] Fix
[noreply] Merge pull request #15991 [BEAM-13251][Playground] [Bugfix] Lint Fails
[noreply] Merge pull request #15990 from [BEAM-13177][Playground] Change using of
[noreply] [BEAM-13262] Forward for metrics.SingleResult (#15993)
[noreply] Generate Python container dependencies in an automated way. (#15927)
[noreply] [BEAM-13264] Allow pyarrow up to 6.x (#15995)
[noreply] Update the go get command with v2 (#15945)
[noreply] Go Quickstart: switch to Spark 3 JobServer for better out-of-box
[noreply] Golint fixes for recent Go SDK import (#15999)
------------------------------------------
[...truncated 18.03 MB...]
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637108223.567763205","description":"Error received from peer ipv4:127.0.0.1:33267","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637108223.567793209","description":"Error received from peer ipv4:127.0.0.1:41329","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:direct:py37:directRunnerIT FAILED
> Task :sdks:python:test-suites:direct:py37:mongodbioIT
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:__main__:Writing 100000 documents to mongodb
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:80: FutureWarning: WriteToMongoDB is experimental.
known_args.batch_size))
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function annotate_downstream_side_inputs at 0x7f28de906560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function fix_side_input_pcoll_coders at 0x7f28de906680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28de906b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f28de906c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_sdf at 0x7f28de906dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_gbk at 0x7f28de906e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sink_flattens at 0x7f28de906f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function greedily_fuse at 0x7f28de907050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function read_to_impulse at 0x7f28de9070e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function impulse_to_input at 0x7f28de907170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f28de9073b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function setup_timer_mapping at 0x7f28de907320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function populate_data_channel_coders at 0x7f28de907440> ====================
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner.worker_handlers:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f28de586a90> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((((ref_AppliedPTransform_Create-Impulse_3)+(ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_Create-Map-decode-_6))+(ref_AppliedPTransform_Create-documents_7))+(ref_AppliedPTransform_WriteToMongoDB-ParDo-_GenerateObjectIdFn-_9))+(ref_AppliedPTransform_WriteToMongoDB-Reshuffle-AddRandomKeys_11))+(ref_AppliedPTransform_WriteToMongoDB-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_13))+(WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((WriteToMongoDB/Reshuffle/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_WriteToMongoDB-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_15))+(ref_AppliedPTransform_WriteToMongoDB-Reshuffle-RemoveRandomKeys_16))+(ref_AppliedPTransform_WriteToMongoDB-ParDo-_WriteMongoFn-_17)
INFO:__main__:Writing 100000 documents to mongodb finished in 27.596 seconds
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:__main__:================================================================================
INFO:__main__:Reading from mongodb beam_mongodbio_it_db:integration_test_1637108246
INFO:__main__:reader params : {'projection': ['number']}
INFO:__main__:expected results: {'number_sum': 4999950000, 'docs_count': 100000}
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/mongodbio_it_test.py>:153: FutureWarning: ReadFromMongoDB is experimental.
| 'Map' >> beam.Map(lambda doc: doc['number']))
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function annotate_downstream_side_inputs at 0x7f28de906560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function fix_side_input_pcoll_coders at 0x7f28de906680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28de906b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f28de906c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_sdf at 0x7f28de906dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_gbk at 0x7f28de906e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sink_flattens at 0x7f28de906f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function greedily_fuse at 0x7f28de907050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function read_to_impulse at 0x7f28de9070e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function impulse_to_input at 0x7f28de907170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f28de9073b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function setup_timer_mapping at 0x7f28de907320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function populate_data_channel_coders at 0x7f28de907440> ====================
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner.worker_handlers:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f28dd72de50> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_ReadFromMongoDB-Read-Impulse_4)+(ref_AppliedPTransform_ReadFromMongoDB-Read-Map-lambda-at-iobase-py-898-_5))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((((((ref_PCollection_PCollection_2_split/Read)+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Map_8))+(ref_AppliedPTransform_Combine-KeyWithVoid_10))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-KeyWithVoid_24))+(Combine/CombinePerKey/Precombine))+(Combine/CombinePerKey/Group/Write))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Precombine))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_39)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-3224-_40))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_42))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_47))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/0))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Read)+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Merge))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-UnKey_29))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Impulse_31)+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-FlatMap-lambda-at-core-py-3224-_32))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Map-decode-_34))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-InjectDefault_35))+(Flatten/Transcode/1))+(Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Combine/CombinePerKey/Group/Read)+(Combine/CombinePerKey/Merge))+(Combine/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Combine-UnKey_15))+(ref_PCollection_PCollection_8/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_Combine-DoOnce-Impulse_17)+(ref_AppliedPTransform_Combine-DoOnce-FlatMap-lambda-at-core-py-3224-_18))+(ref_AppliedPTransform_Combine-DoOnce-Map-decode-_20))+(ref_AppliedPTransform_Combine-InjectDefault_21))+(Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((Flatten/Read)+(ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_43))+(ref_AppliedPTransform_assert_that-ToVoidKey_44))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-1-_48))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/1))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (assert_that/Group/CoGroupByKeyImpl/Flatten/Read)+(assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_51))+(ref_AppliedPTransform_assert_that-Group-RestoreTags_52))+(ref_AppliedPTransform_assert_that-Unkey_53))+(ref_AppliedPTransform_assert_that-Match_54)
INFO:__main__:Reading documents from mongodb finished in 5.197 seconds
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:__main__:================================================================================
INFO:__main__:Reading from mongodb beam_mongodbio_it_db:integration_test_1637108246
INFO:__main__:reader params : {'filter': {'number_mod_3': 0}, 'projection': ['number']}
INFO:__main__:expected results: {'number_sum': 1666683333, 'docs_count': 33334}
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function annotate_downstream_side_inputs at 0x7f28de906560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function fix_side_input_pcoll_coders at 0x7f28de906680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28de906b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f28de906c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_sdf at 0x7f28de906dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_gbk at 0x7f28de906e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sink_flattens at 0x7f28de906f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function greedily_fuse at 0x7f28de907050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function read_to_impulse at 0x7f28de9070e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function impulse_to_input at 0x7f28de907170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f28de9073b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function setup_timer_mapping at 0x7f28de907320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function populate_data_channel_coders at 0x7f28de907440> ====================
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner.worker_handlers:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f28dc3f8490> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_ReadFromMongoDB-Read-Impulse_4)+(ref_AppliedPTransform_ReadFromMongoDB-Read-Map-lambda-at-iobase-py-898-_5))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((((((ref_PCollection_PCollection_2_split/Read)+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Map_8))+(ref_AppliedPTransform_Combine-KeyWithVoid_10))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-KeyWithVoid_24))+(Combine/CombinePerKey/Precombine))+(Combine/CombinePerKey/Group/Write))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Precombine))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Read)+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Merge))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-UnKey_29))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_39)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-3224-_40))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_42))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_47))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/0))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Combine/CombinePerKey/Group/Read)+(Combine/CombinePerKey/Merge))+(Combine/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Combine-UnKey_15))+(ref_PCollection_PCollection_8/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Impulse_31)+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-FlatMap-lambda-at-core-py-3224-_32))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Map-decode-_34))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-InjectDefault_35))+(Flatten/Transcode/1))+(Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_Combine-DoOnce-Impulse_17)+(ref_AppliedPTransform_Combine-DoOnce-FlatMap-lambda-at-core-py-3224-_18))+(ref_AppliedPTransform_Combine-DoOnce-Map-decode-_20))+(ref_AppliedPTransform_Combine-InjectDefault_21))+(Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((Flatten/Read)+(ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_43))+(ref_AppliedPTransform_assert_that-ToVoidKey_44))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-1-_48))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/1))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (assert_that/Group/CoGroupByKeyImpl/Flatten/Read)+(assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_51))+(ref_AppliedPTransform_assert_that-Group-RestoreTags_52))+(ref_AppliedPTransform_assert_that-Unkey_53))+(ref_AppliedPTransform_assert_that-Match_54)
INFO:__main__:Reading documents from mongodb finished in 2.644 seconds
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:__main__:================================================================================
INFO:__main__:Reading from mongodb beam_mongodbio_it_db:integration_test_1637108246
INFO:__main__:reader params : {'projection': ['number'], 'bucket_auto': True}
INFO:__main__:expected results: {'number_sum': 4999950000, 'docs_count': 100000}
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function annotate_downstream_side_inputs at 0x7f28de906560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function fix_side_input_pcoll_coders at 0x7f28de906680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28de906b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f28de906c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_sdf at 0x7f28de906dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_gbk at 0x7f28de906e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sink_flattens at 0x7f28de906f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function greedily_fuse at 0x7f28de907050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function read_to_impulse at 0x7f28de9070e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function impulse_to_input at 0x7f28de907170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f28de9073b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function setup_timer_mapping at 0x7f28de907320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function populate_data_channel_coders at 0x7f28de907440> ====================
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner.worker_handlers:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f28dd2adf90> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_ReadFromMongoDB-Read-Impulse_4)+(ref_AppliedPTransform_ReadFromMongoDB-Read-Map-lambda-at-iobase-py-898-_5))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((((((ref_PCollection_PCollection_2_split/Read)+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Map_8))+(ref_AppliedPTransform_Combine-KeyWithVoid_10))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-KeyWithVoid_24))+(Combine/CombinePerKey/Precombine))+(Combine/CombinePerKey/Group/Write))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Precombine))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Read)+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Merge))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-UnKey_29))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Impulse_31)+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-FlatMap-lambda-at-core-py-3224-_32))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Map-decode-_34))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-InjectDefault_35))+(Flatten/Transcode/1))+(Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Combine/CombinePerKey/Group/Read)+(Combine/CombinePerKey/Merge))+(Combine/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Combine-UnKey_15))+(ref_PCollection_PCollection_8/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_Combine-DoOnce-Impulse_17)+(ref_AppliedPTransform_Combine-DoOnce-FlatMap-lambda-at-core-py-3224-_18))+(ref_AppliedPTransform_Combine-DoOnce-Map-decode-_20))+(ref_AppliedPTransform_Combine-InjectDefault_21))+(Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((Flatten/Read)+(ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_43))+(ref_AppliedPTransform_assert_that-ToVoidKey_44))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-1-_48))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/1))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_39)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-3224-_40))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_42))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_47))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/0))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (assert_that/Group/CoGroupByKeyImpl/Flatten/Read)+(assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_51))+(ref_AppliedPTransform_assert_that-Group-RestoreTags_52))+(ref_AppliedPTransform_assert_that-Unkey_53))+(ref_AppliedPTransform_assert_that-Match_54)
INFO:__main__:Reading documents from mongodb finished in 5.206 seconds
INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO:__main__:================================================================================
INFO:__main__:Reading from mongodb beam_mongodbio_it_db:integration_test_1637108246
INFO:__main__:reader params : {'filter': {'number_mod_3': 0}, 'projection': ['number'], 'bucket_auto': True}
INFO:__main__:expected results: {'number_sum': 1666683333, 'docs_count': 33334}
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function annotate_downstream_side_inputs at 0x7f28de906560> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function fix_side_input_pcoll_coders at 0x7f28de906680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f28de906b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f28de906c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_sdf at 0x7f28de906dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function expand_gbk at 0x7f28de906e60> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sink_flattens at 0x7f28de906f80> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function greedily_fuse at 0x7f28de907050> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function read_to_impulse at 0x7f28de9070e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function impulse_to_input at 0x7f28de907170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f28de9073b0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function setup_timer_mapping at 0x7f28de907320> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function populate_data_channel_coders at 0x7f28de907440> ====================
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner.worker_handlers:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f28de515a50> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_39)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-3224-_40))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_42))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-0-_47))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/0))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_ReadFromMongoDB-Read-Impulse_4)+(ref_AppliedPTransform_ReadFromMongoDB-Read-Map-lambda-at-iobase-py-898-_5))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((((((ref_PCollection_PCollection_2_split/Read)+(ReadFromMongoDB/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Map_8))+(ref_AppliedPTransform_Combine-KeyWithVoid_10))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-KeyWithVoid_24))+(Combine/CombinePerKey/Precombine))+(Combine/CombinePerKey/Group/Write))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Precombine))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Combine/CombinePerKey/Group/Read)+(Combine/CombinePerKey/Merge))+(Combine/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Combine-UnKey_15))+(ref_PCollection_PCollection_8/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((ref_AppliedPTransform_Combine-DoOnce-Impulse_17)+(ref_AppliedPTransform_Combine-DoOnce-FlatMap-lambda-at-core-py-3224-_18))+(ref_AppliedPTransform_Combine-DoOnce-Map-decode-_20))+(ref_AppliedPTransform_Combine-InjectDefault_21))+(Flatten/Write/0)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((Count/CombineGlobally(CountCombineFn)/CombinePerKey/Group/Read)+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/Merge))+(Count/CombineGlobally(CountCombineFn)/CombinePerKey/ExtractOutputs))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-UnKey_29))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Impulse_31)+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-FlatMap-lambda-at-core-py-3224-_32))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-DoOnce-Map-decode-_34))+(ref_AppliedPTransform_Count-CombineGlobally-CountCombineFn-InjectDefault_35))+(Flatten/Transcode/1))+(Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((Flatten/Read)+(ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_43))+(ref_AppliedPTransform_assert_that-ToVoidKey_44))+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-Tag-1-_48))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Transcode/1))+(assert_that/Group/CoGroupByKeyImpl/Flatten/Write/1)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (assert_that/Group/CoGroupByKeyImpl/Flatten/Read)+(assert_that/Group/CoGroupByKeyImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((((assert_that/Group/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-CoGroupByKeyImpl-MapTuple-collect_values-_51))+(ref_AppliedPTransform_assert_that-Group-RestoreTags_52))+(ref_AppliedPTransform_assert_that-Unkey_53))+(ref_AppliedPTransform_assert_that-Match_54)
INFO:__main__:Reading documents from mongodb finished in 2.716 seconds
> Task :sdks:python:test-suites:direct:py37:postCommitIT
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 112
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:directRunnerIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 36m 34s
214 actionable tasks: 153 executed, 57 from cache, 4 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (1 retry remaining)...
A network error occurred.
If you require assistance with this problem, please report it via https://gradle.com/help/plugin and include the following information via copy/paste.
----------
Gradle version: 6.9.1
Plugin version: 3.4.1
Request URL: https://scans-in.gradle.com/scans/publish/gradle/3.4.1/token
Request ID: 32cbf699-5ef2-453b-955e-47e9c9e5af21
Exception: java.net.SocketTimeoutException: Read timed out
----------
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4524
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4524/display/redirect?page=changes>
Changes:
[noreply] [BEAM-3293] Finish E2E implementation of MultiMap side inputs, add
------------------------------------------
[...truncated 61.42 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 6646.81 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 0s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ylwimsqs5oedu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4523
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4523/display/redirect>
Changes:
------------------------------------------
[...truncated 47.87 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3224-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 230764150
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 335561752
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 349464416
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 349648237
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 349732875
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 349800586
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 350286960
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637065437
nanos: 350405693
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.192802906036377 seconds.
INFO:root:Successfully completed job in 6.192802906036377 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39579
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f32dbc5bd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f32dbc5bdd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f32dbc5f560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temput06nx0_/artifactsugpc1bp_' '--job-port' '47811' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42265'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:01 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39203'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:47811'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_ef33f494-000f-4e72-a452-7b53b10efc1a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_ef33f494-000f-4e72-a452-7b53b10efc1a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_ef33f494-000f-4e72-a452-7b53b10efc1a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_ef33f494-000f-4e72-a452-7b53b10efc1a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:02 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1116122402-bed3f0d6_e5b50239-e0f6-4477-8d3e-904ab18f6bcc'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:03 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1116122402-bed3f0d6_e5b50239-e0f6-4477-8d3e-904ab18f6bcc'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:03 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:03 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:04 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:04 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1116122402-bed3f0d6_e5b50239-e0f6-4477-8d3e-904ab18f6bcc on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42273.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44357.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46345
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:05 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1116122402-bed3f0d6_e5b50239-e0f6-4477-8d3e-904ab18f6bcc: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/16 12:24:06 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1116122402-bed3f0d6_e5b50239-e0f6-4477-8d3e-904ab18f6bcc finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637065447.311256566","description":"Error received from peer ipv4:127.0.0.1:44357","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637065447.311255430","description":"Error received from peer ipv4:127.0.0.1:46345","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637065447.311255430","description":"Error received from peer ipv4:127.0.0.1:46345","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637065447.311269547","description":"Error received from peer ipv4:127.0.0.1:42273","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 56m 55s
214 actionable tasks: 159 executed, 51 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ydbana7eqv6n2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4522
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4522/display/redirect?page=changes>
Changes:
[brachipa] [BEAM-12737] error handling for sql calc error
[brachipa] [BEAM-12737] error handling for sql calc error
[brachipa] [BEAM-12737] error handling for sql calc error
[brachipa] [BEAM-12737] add test
[brachipa] [BEAM-12737] fix build
[brachipa] [BEAM-12737] fix build
[brachipa] revert
[brachipa] [beam-12737] cr fix
[brachipa] [beam-12737] cr fixes
[brachipa] [beam-12737] send errorTransformer to `buildPTransform`
[brachipa] [beam-12737] send errorTransformer to `buildPTransform`
[brachipa] [beam-12737] fix type `? extends POutput`
[brachipa] [beam-12737] add complex test, change error row type
[noreply] [BEAM-13221] Don't drop last data. (#15939)
------------------------------------------
[...truncated 45.48 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3224-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 315691232
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 454187870
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 460128545
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 460295677
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 460368394
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 460438489
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 461440563
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1637043549
nanos: 461591005
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.219338417053223 seconds.
INFO:root:Successfully completed job in 8.219338417053223 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44623
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f433386ed40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f433386edd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4333871560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempjt0ajkhf/artifactsxz4qkjqz' '--job-port' '51777' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:33471'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40487'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:51777'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_25272820-0a67-4c94-bea0-b2cc45723eb8.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_25272820-0a67-4c94-bea0-b2cc45723eb8.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_25272820-0a67-4c94-bea0-b2cc45723eb8.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:14 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_25272820-0a67-4c94-bea0-b2cc45723eb8.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:15 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1116061915-a4649769_2019849b-9f3f-4726-a360-5f72d405aa0b'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:15 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1116061915-a4649769_2019849b-9f3f-4726-a360-5f72d405aa0b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:15 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:15 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:16 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:16 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:16 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1116061915-a4649769_2019849b-9f3f-4726-a360-5f72d405aa0b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33247.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:45803.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35793
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:17 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1116061915-a4649769_2019849b-9f3f-4726-a360-5f72d405aa0b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/16 06:19:18 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1116061915-a4649769_2019849b-9f3f-4726-a360-5f72d405aa0b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637043559.473693769","description":"Error received from peer ipv4:127.0.0.1:35793","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637043559.473693769","description":"Error received from peer ipv4:127.0.0.1:35793","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637043559.473721808","description":"Error received from peer ipv4:127.0.0.1:45803","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1637043559.473721805","description":"Error received from peer ipv4:127.0.0.1:33247","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 45s
214 actionable tasks: 172 executed, 38 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2cy7m3sh7cshq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4521 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4521 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4521/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4520
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4520/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13226] Batch metrics parsing errors (#15972)
[noreply] [BEAM-13239] pass through io.EOF when decoding row (#15971)
[noreply] [BEAM-5378] Register types & funcs for examples. (#15970)
------------------------------------------
[...truncated 47.32 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [4]assert_that/{Create, Group} (12/16) (89d832a5566188cfbe2f48f7135c5267) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [4]assert_that/{Create, Group} (1/16) (bf986575a985429bdde54c83395387cf) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [4]assert_that/{Create, Group} (7/16) (7c46d37bc5abe0524c4d1c0d74bfcb96) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [4]assert_that/{Create, Group} (5/16)#0 (d05062e35d891a2752e14038f49d4ce4) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Freeing task resources for [4]assert_that/{Create, Group} (5/16)#0 (d05062e35d891a2752e14038f49d4ce4).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task [4]assert_that/{Create, Group} (5/16)#0 d05062e35d891a2752e14038f49d4ce4.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:45 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [4]assert_that/{Create, Group} (5/16) (d05062e35d891a2752e14038f49d4ce4) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:42:55 PM org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory createEnvironment'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Still waiting for startup of environment apache/beam_java8_sdk:2.35.0.dev for worker id 1-1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:00 PM org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory createEnvironment'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Still waiting for startup of environment apache/beam_java8_sdk:2.35.0.dev for worker id 1-1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService control'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Beam Fn Control client connected with id 1-1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.beam.runners.fnexecution.logging.GrpcLoggingService logging'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Beam Fn Logging client connected.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16)#0 (fbe49ad3a504eb7a618de29be8abef78) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16) (fbe49ad3a504eb7a618de29be8abef78) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Fn Harness started'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter log'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Entering instruction processing loop'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (10/16)#0 (053f43baf1adeec3e891467a76b2202a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (10/16) (053f43baf1adeec3e891467a76b2202a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (16/16)#0 (30fe2753991878b216efe65f9ae66b8c) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (16/16) (30fe2753991878b216efe65f9ae66b8c) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (12/16)#0 (da5310ac91952ae2495e2e9e60a16cfb) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (12/16) (da5310ac91952ae2495e2e9e60a16cfb) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (12/16)#0 (e7484b8ff7ee51cee595bab21e528b93) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (15/16)#0 (1a032448a7b137ba23fcbf8522ace952) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (12/16) (e7484b8ff7ee51cee595bab21e528b93) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (15/16) (1a032448a7b137ba23fcbf8522ace952) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (14/16)#0 (5c526411b991efa7499293f252f7a627) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (14/16) (5c526411b991efa7499293f252f7a627) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (5/16)#0 (92089115d5b0d4f1b704389b7c6a3444) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (7/16)#0 (31d8c292dc775694928215b7ed4c4441) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (5/16) (92089115d5b0d4f1b704389b7c6a3444) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (7/16) (31d8c292dc775694928215b7ed4c4441) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (7/16)#0 (7056df01267d7e7c951310ab3734189d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (1/16)#0 (3e3dbbf1db4535b204cead4a79188e9f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (7/16) (7056df01267d7e7c951310ab3734189d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (1/16) (3e3dbbf1db4535b204cead4a79188e9f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (12/16)#0 (2dc7b60dc9823955848eddf2d17aafef) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (12/16) (2dc7b60dc9823955848eddf2d17aafef) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (3/16)#0 (24bb74d3d55b13689a124a41dc91c3ec) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (3/16) (24bb74d3d55b13689a124a41dc91c3ec) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (13/16)#0 (8a3241073be6d5c47644d89d9f0dea15) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (13/16) (8a3241073be6d5c47644d89d9f0dea15) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (10/16)#0 (fbda437a5750402ff96e3238d939a092) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (5/16)#0 (9681fdf85b80b3d9c394bf5cffef1a8d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (16/16)#0 (996bdeea6f78da165d5485f8c8175b9f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (6/16)#0 (c8b5bb6dfa65ab5d2eb345a1cbf1b5e9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (9/16)#0 (15590e13824493a18545aa408270b288) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (13/16)#0 (c79f6f927862820458da53ea43673e13) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (8/16)#0 (8b50552d8ee7d62bf4e47018c86d0447) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (11/16)#0 (b319dae88aa0ac51da8e67b5b43afc30) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (1/16)#0 (46507acdedaf2cd8ec6395a30d19cdc8) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (2/16)#0 (ea6489386001f223b230f1ec8231312b) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (16/16)#0 (c35ebcaa100932f9ff6203c463e67acd) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (8/16)#0 (089b0fb542a62924bb37c6e6924bb1a6) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (2/16)#0 (c27dc04d93b4770eb5b8772598d6b963) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (6/16)#0 (f5a042474f50484bbd961953ba1d3410) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (3/16)#0 (b4448f037913bf21aceacf3db51d00be) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (13/16)#0 (8f4575449cd327e50251abe3b9777930) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (4/16)#0 (3d859a82e4fc9d7f1559eb011be1d653) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (11/16)#0 (009fcd5e9e7490d65a847de4864d4915) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (7/16)#0 (39821f85b997e7ca1dc8c1a7b050be68) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (4/16)#0 (3a8a00d6fa3eef81110c184e2463bd2a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (15/16)#0 (b5e83f32951df8d75de0f73c14f9a51d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (10/16) (fbda437a5750402ff96e3238d939a092) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (5/16) (9681fdf85b80b3d9c394bf5cffef1a8d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (16/16) (996bdeea6f78da165d5485f8c8175b9f) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (6/16) (c8b5bb6dfa65ab5d2eb345a1cbf1b5e9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (9/16) (15590e13824493a18545aa408270b288) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (13/16) (c79f6f927862820458da53ea43673e13) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (8/16) (8b50552d8ee7d62bf4e47018c86d0447) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (11/16) (b319dae88aa0ac51da8e67b5b43afc30) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (1/16) (46507acdedaf2cd8ec6395a30d19cdc8) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (6/16)#0 (16e5e0492ef6a013c37d758009f11986) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (2/16) (ea6489386001f223b230f1ec8231312b) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (16/16) (c35ebcaa100932f9ff6203c463e67acd) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (8/16) (089b0fb542a62924bb37c6e6924bb1a6) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (2/16) (c27dc04d93b4770eb5b8772598d6b963) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (6/16) (f5a042474f50484bbd961953ba1d3410) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (3/16) (b4448f037913bf21aceacf3db51d00be) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (13/16) (8f4575449cd327e50251abe3b9777930) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (4/16) (3d859a82e4fc9d7f1559eb011be1d653) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (11/16) (009fcd5e9e7490d65a847de4864d4915) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (7/16) (39821f85b997e7ca1dc8c1a7b050be68) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [1]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/ParDo(KafkaSourceConsumer)/ParMultiDo(KafkaSourceConsumer)/ProcessSizedElementsAndRestrictions0 -> [5]{Read from debezium, assert_that} (4/16) (3a8a00d6fa3eef81110c184e2463bd2a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (15/16) (b5e83f32951df8d75de0f73c14f9a51d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (6/16) (16e5e0492ef6a013c37d758009f11986) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (9/16)#0 (8a0e76e8eede8fc7b6d9fd39722a07cd) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (9/16) (8a0e76e8eede8fc7b6d9fd39722a07cd) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (1/16)#0 (c5f350ca07880fcc23b1324d7527d309) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_io_read_it_test.py::BigqueryIOReadIT::test_bigquery_read_custom_1M_python
apache_beam/io/gcp/pubsub_integration_test.py::PubSubIntegrationTest::test_streaming_data_only
> Task :sdks:python:test-suites:portable:py37:postCommitPy37IT
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (1/16) (c5f350ca07880fcc23b1324d7527d309) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (8/16)#0 (be88d087da54c5a85fdfa311056fef3a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (8/16) (be88d087da54c5a85fdfa311056fef3a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (4/16)#0 (4eede53050256b434ce1bde152c8fa6d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (9/16)#0 (109c377c513abbdc0e4fc62242c0544a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (4/16) (4eede53050256b434ce1bde152c8fa6d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (9/16) (109c377c513abbdc0e4fc62242c0544a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (2/16)#0 (81ce93d912521b5f878224771b606199) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (2/16) (81ce93d912521b5f878224771b606199) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (3/16)#0 (3d65fe6d39f327060c85ba7ef9f65ff9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (3/16) (3d65fe6d39f327060c85ba7ef9f65ff9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16)#0 (fbe49ad3a504eb7a618de29be8abef78) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Freeing task resources for [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16)#0 (fbe49ad3a504eb7a618de29be8abef78).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskexecutor.TaskExecutor unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Un-registering task and sending final execution state FINISHED to JobManager for task [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16)#0 fbe49ad3a504eb7a618de29be8abef78.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16) (fbe49ad3a504eb7a618de29be8abef78) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (10/16)#0 (26020facdb542a2c633f79bd8d966b77) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (10/16) (26020facdb542a2c633f79bd8d966b77) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (2/16)#0 (c27dc04d93b4770eb5b8772598d6b963) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: Freeing task resources for [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (2/16)#0 (c27dc04d93b4770eb5b8772598d6b963).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:04 PM org.apache.flink.runtime.taskmanager.Task transitionState'
> Task :sdks:python:test-suites:portable:py37:postCommitPy37IT
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (1/16) (c5f350ca07880fcc23b1324d7527d309) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (8/16)#0 (be88d087da54c5a85fdfa311056fef3a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (8/16) (be88d087da54c5a85fdfa311056fef3a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (4/16)#0 (4eede53050256b434ce1bde152c8fa6d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (9/16)#0 (109c377c513abbdc0e4fc62242c0544a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (4/16) (4eede53050256b434ce1bde152c8fa6d) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (9/16) (109c377c513abbdc0e4fc62242c0544a) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (2/16)#0 (81ce93d912521b5f878224771b606199) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/{Create.Values, ParDo(KafkaSourceConsumer)} (2/16) (81ce93d912521b5f878224771b606199) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (3/16)#0 (3d65fe6d39f327060c85ba7ef9f65ff9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.executiongraph.Execution transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (3/16) (3d65fe6d39f327060c85ba7ef9f65ff9) switched from INITIALIZING to RUNNING.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'INFO: [3]Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)/Create.Values/Read(CreateSource)/{ParDo(OutputSingleSource), ParDo(BoundedSourceAsSDFWrapper)} (15/16)#0 (fbe49ad3a504eb7a618de29be8abef78) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:122 b'Nov 15, 2021 6:43:03 PM org.apache.flink.runtime.taskmanager.Task doRun'
java.lang.OutOfMemoryError: GC overhead limit exceeded
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4519 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4519 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4519/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4518
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4518/display/redirect>
Changes:
------------------------------------------
[...truncated 17.83 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 503244876
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 509301900
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 509464025
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 509544849
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 509611606
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 510003089
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636957005
nanos: 510097265
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.227110147476196 seconds.
INFO:root:Successfully completed job in 7.227110147476196 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33681
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f52fb4195f0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f52fb419680> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f52fb419dd0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempda_z6qwi/artifactsc3k1tinj' '--job-port' '43949' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46821'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42157'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:43949'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1c5c4353-b9c4-4b5e-8cdf-5445be03e10e.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1c5c4353-b9c4-4b5e-8cdf-5445be03e10e.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1c5c4353-b9c4-4b5e-8cdf-5445be03e10e.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1c5c4353-b9c4-4b5e-8cdf-5445be03e10e.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1115061651-8373a92d_6940b3ee-8a6d-4920-acb0-5fd218a06b6f'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:51 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1115061651-8373a92d_6940b3ee-8a6d-4920-acb0-5fd218a06b6f'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:52 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:52 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:53 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:53 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1115061651-8373a92d_6940b3ee-8a6d-4920-acb0-5fd218a06b6f on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36933.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46361.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33645
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1115061651-8373a92d_6940b3ee-8a6d-4920-acb0-5fd218a06b6f: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:55 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/15 06:16:56 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1115061651-8373a92d_6940b3ee-8a6d-4920-acb0-5fd218a06b6f finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636957016.609410822","description":"Error received from peer ipv4:127.0.0.1:33645","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636957016.609423266","description":"Error received from peer ipv4:127.0.0.1:46361","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636957016.609410822","description":"Error received from peer ipv4:127.0.0.1:33645","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636957016.609428696","description":"Error received from peer ipv4:127.0.0.1:36933","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 36m 33s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3suq7s3hiyb7a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4517
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4517/display/redirect>
Changes:
------------------------------------------
[...truncated 42.08 MB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3222-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636936007
nanos: 867840528
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636936007
nanos: 998670339
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 12922763
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 13129234
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 13221740
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 13295650
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 14126062
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636936008
nanos: 14242649
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.822757720947266 seconds.
INFO:root:Successfully completed job in 6.822757720947266 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40169
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f0ae44a4d40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f0ae44a4dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f0ae44a6560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempt7nw5v4e/artifactsiapsksqv' '--job-port' '37685' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:54 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:40847'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34851'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:37685'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:55 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:37685.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_07ba57a4-356c-4c37-b063-5aab6574bc61.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_07ba57a4-356c-4c37-b063-5aab6574bc61.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_07ba57a4-356c-4c37-b063-5aab6574bc61.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_07ba57a4-356c-4c37-b063-5aab6574bc61.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:56 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1115002656-bb5da0e2_3240fe36-7a53-4f68-8935-07d3836a6847'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:57 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1115002656-bb5da0e2_3240fe36-7a53-4f68-8935-07d3836a6847'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:26:59 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:01 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:01 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:01 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1115002656-bb5da0e2_3240fe36-7a53-4f68-8935-07d3836a6847 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:45997.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40641.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:40785
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:03 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1115002656-bb5da0e2_3240fe36-7a53-4f68-8935-07d3836a6847: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/15 00:27:04 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1115002656-bb5da0e2_3240fe36-7a53-4f68-8935-07d3836a6847 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636936024.846743408","description":"Error received from peer ipv4:127.0.0.1:40785","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636936024.846764219","description":"Error received from peer ipv4:127.0.0.1:45997","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636936024.846744363","description":"Error received from peer ipv4:127.0.0.1:40641","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636936024.846743408","description":"Error received from peer ipv4:127.0.0.1:40785","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 31s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/nk6lnkd72wpoy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4516
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4516/display/redirect>
Changes:
------------------------------------------
[...truncated 510.32 KB...]
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 639235734
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 646893024
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 647055625
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 647134304
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 647199392
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 648737192
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636914044
nanos: 648874998
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.400390863418579 seconds.
INFO:root:Successfully completed job in 7.400390863418579 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:44453
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fc2f437bd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fc2f437bdd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fc2f4376560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temph3a321_p/artifactsxyfalx9z' '--job-port' '40567' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36383'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35035'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40567'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:49 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_cf3319f0-4f58-4192-9fc0-1938516942e1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_cf3319f0-4f58-4192-9fc0-1938516942e1.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_cf3319f0-4f58-4192-9fc0-1938516942e1.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_cf3319f0-4f58-4192-9fc0-1938516942e1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1114182050-4ec256db_ca2ec6e2-0347-4096-8989-7eb5b53e607c'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1114182050-4ec256db_ca2ec6e2-0347-4096-8989-7eb5b53e607c'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:50 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:51 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:52 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:52 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1114182050-4ec256db_ca2ec6e2-0347-4096-8989-7eb5b53e607c on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39713.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35797.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33185
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114182050-4ec256db_ca2ec6e2-0347-4096-8989-7eb5b53e607c: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:54 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 18:20:54 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114182050-4ec256db_ca2ec6e2-0347-4096-8989-7eb5b53e607c finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636914054.609284529","description":"Error received from peer ipv4:127.0.0.1:35797","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636914054.609290331","description":"Error received from peer ipv4:127.0.0.1:39713","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636914054.609249026","description":"Error received from peer ipv4:127.0.0.1:33185","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636914054.609249026","description":"Error received from peer ipv4:127.0.0.1:33185","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 7s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/lsjuxn6fynfwm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4515
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4515/display/redirect>
Changes:
------------------------------------------
[...truncated 14.45 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 6602.76 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 24s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ttz7lszkw3rk6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4514
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4514/display/redirect>
Changes:
------------------------------------------
[...truncated 40.84 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 106621503
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 278392553
}
message: "Renamed 1 shards in 0.17 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 319668292
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 319843053
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 319921731
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 319991350
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 320372343
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636870948
nanos: 320468425
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 4.676674127578735 seconds.
INFO:root:Successfully completed job in 4.676674127578735 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:41341
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3edfa9cd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3edfa9cdd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3edfa98560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempjq0o50to/artifacts3wjvefyb' '--job-port' '36789' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37377'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44339'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36789'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:32 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_d6334798-3247-419d-8127-0779bdf60f53.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_d6334798-3247-419d-8127-0779bdf60f53.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_d6334798-3247-419d-8127-0779bdf60f53.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_d6334798-3247-419d-8127-0779bdf60f53.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:34 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1114062233-7b23f595_0d064d8e-4574-4af4-8df3-676d48c91751'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:34 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1114062233-7b23f595_0d064d8e-4574-4af4-8df3-676d48c91751'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:34 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:34 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:35 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:35 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:35 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1114062233-7b23f595_0d064d8e-4574-4af4-8df3-676d48c91751 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43143.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36013.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36369
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114062233-7b23f595_0d064d8e-4574-4af4-8df3-676d48c91751: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 06:22:37 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114062233-7b23f595_0d064d8e-4574-4af4-8df3-676d48c91751 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636870958.043924826","description":"Error received from peer ipv4:127.0.0.1:43143","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636870958.043876162","description":"Error received from peer ipv4:127.0.0.1:36369","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636870958.043924880","description":"Error received from peer ipv4:127.0.0.1:36013","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636870958.043876162","description":"Error received from peer ipv4:127.0.0.1:36369","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 49s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vr2uex4bnle6m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4513
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4513/display/redirect>
Changes:
------------------------------------------
[...truncated 397.24 KB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 680855512
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 785301208
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 799683094
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 799883604
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 800013065
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 800121784
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 800620079
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636849123
nanos: 800742149
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 4.7741851806640625 seconds.
INFO:root:Successfully completed job in 4.7741851806640625 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34227
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fee5b1ced40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fee5b1cedd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fee5b1cf560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempyvxwno1n/artifacts2itt2o4r' '--job-port' '40537' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:47 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38335'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44059'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:40537'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_31531ae5-2a7b-4f80-88f1-c42b220bd716.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_31531ae5-2a7b-4f80-88f1-c42b220bd716.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_31531ae5-2a7b-4f80-88f1-c42b220bd716.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:48 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_31531ae5-2a7b-4f80-88f1-c42b220bd716.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:49 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1114001849-3d5940e5_babfe858-8d87-4bc0-a657-30b8744a3916'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:49 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1114001849-3d5940e5_babfe858-8d87-4bc0-a657-30b8744a3916'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:49 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:49 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:50 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:50 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1114001849-3d5940e5_babfe858-8d87-4bc0-a657-30b8744a3916 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:42225.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41553.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:36009
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114001849-3d5940e5_babfe858-8d87-4bc0-a657-30b8744a3916: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/14 00:18:52 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1114001849-3d5940e5_babfe858-8d87-4bc0-a657-30b8744a3916 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636849133.132913863","description":"Error received from peer ipv4:127.0.0.1:36009","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636849133.132913863","description":"Error received from peer ipv4:127.0.0.1:36009","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636849133.132937227","description":"Error received from peer ipv4:127.0.0.1:42225","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636849133.132926863","description":"Error received from peer ipv4:127.0.0.1:41553","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 58s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iajlu223ajzky
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4512
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4512/display/redirect>
Changes:
------------------------------------------
[...truncated 51.17 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 178 warnings in 6621.18 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 58m 1s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/kzvqzvt4qnjsm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4511
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4511/display/redirect>
Changes:
------------------------------------------
[...truncated 45.68 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 196168422
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 335022687
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340235710
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340379238
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340453147
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340516805
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340889930
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636805900
nanos: 340987443
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.1147301197052 seconds.
INFO:root:Successfully completed job in 5.1147301197052 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33267
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3db4fbbd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3db4fbbdd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3db4fb6560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temphryvtkn5/artifactsgtu21xq_' '--job-port' '44813' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45867'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39697'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44813'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:24 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_25d2d19c-60f6-414e-92df-ff555b7d8944.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_25d2d19c-60f6-414e-92df-ff555b7d8944.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_25d2d19c-60f6-414e-92df-ff555b7d8944.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_25d2d19c-60f6-414e-92df-ff555b7d8944.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1113121825-4cfbdfca_763528cc-bb4e-4edf-9921-e88319f9e79f'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:25 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1113121825-4cfbdfca_763528cc-bb4e-4edf-9921-e88319f9e79f'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:26 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:26 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:27 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:27 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1113121825-4cfbdfca_763528cc-bb4e-4edf-9921-e88319f9e79f on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35831.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:46735.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43259
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:28 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113121825-4cfbdfca_763528cc-bb4e-4edf-9921-e88319f9e79f: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 12:18:29 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113121825-4cfbdfca_763528cc-bb4e-4edf-9921-e88319f9e79f finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636805909.862235730","description":"Error received from peer ipv4:127.0.0.1:35831","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636805909.862205699","description":"Error received from peer ipv4:127.0.0.1:46735","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636805909.862200918","description":"Error received from peer ipv4:127.0.0.1:43259","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636805909.862200918","description":"Error received from peer ipv4:127.0.0.1:43259","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 22s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/2fbhasnhmbltu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4510
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4510/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13223] Documents two new features of cross-language transforms
------------------------------------------
[...truncated 37.07 MB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3222-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636784847
nanos: 840327501
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636784847
nanos: 988754034
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 16214847
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 16454219
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 16550540
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 16666889
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 17929077
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636784848
nanos: 18135786
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.9070045948028564 seconds.
INFO:root:Successfully completed job in 7.9070045948028564 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35191
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff9e27dfd40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff9e27dfdd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff9e27e0560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp0g44b9vj/artifacts7vlue_8t' '--job-port' '39659' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46017'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:32901'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39659'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:35 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:39659.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_87279fd5-71ab-428a-9ae0-60c20aadb8f3.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_87279fd5-71ab-428a-9ae0-60c20aadb8f3.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_87279fd5-71ab-428a-9ae0-60c20aadb8f3.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:37 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_87279fd5-71ab-428a-9ae0-60c20aadb8f3.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:37 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1113062737-c6008392_82e05c91-d63d-4ca5-a6fd-76ab1c83075a'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:37 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1113062737-c6008392_82e05c91-d63d-4ca5-a6fd-76ab1c83075a'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:37 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:37 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:38 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:38 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:38 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1113062737-c6008392_82e05c91-d63d-4ca5-a6fd-76ab1c83075a on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38805.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:39 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40761.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42629
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:39 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113062737-c6008392_82e05c91-d63d-4ca5-a6fd-76ab1c83075a: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 06:27:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113062737-c6008392_82e05c91-d63d-4ca5-a6fd-76ab1c83075a finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636784861.463667813","description":"Error received from peer ipv4:127.0.0.1:38805","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636784861.463630297","description":"Error received from peer ipv4:127.0.0.1:40761","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636784861.463629237","description":"Error received from peer ipv4:127.0.0.1:42629","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636784861.463629237","description":"Error received from peer ipv4:127.0.0.1:42629","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 42m 30s
214 actionable tasks: 158 executed, 52 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/e2lk4lbhdwy7w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4509
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4509/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if
[mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if
[aydar.zaynutdinov] [BEAM-13224][Playground]
[noreply] [BEAM-13193] Add elements field in ProcessBundleRequest and ProcessBu…
[noreply] [BEAM-3304] Refactor trigger API (#15952)
[noreply] [BEAM-3293] Add NewKeyedIterable function to SideInputAdapter interface
------------------------------------------
[...truncated 1.82 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 239045381
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 388182640
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 414269924
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 414504766
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 414631843
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 414734125
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 416292667
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636763572
nanos: 416473150
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 18.29801321029663 seconds.
INFO:root:Successfully completed job in 18.29801321029663 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:42049
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fe770781d40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fe770781dd0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fe77077d560> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp77zu2b0y/artifactsm24zxu__' '--job-port' '52289' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:52289.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:03 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:45535'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:35181'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:52289'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:04 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:52289.
WARNING:root:Waiting for grpc channel to be ready at localhost:52289.
WARNING:root:Waiting for grpc channel to be ready at localhost:52289.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_042f34d1-812e-453a-9244-47cdac238b84.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_042f34d1-812e-453a-9244-47cdac238b84.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_042f34d1-812e-453a-9244-47cdac238b84.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:09 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_042f34d1-812e-453a-9244-47cdac238b84.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:10 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1113003310-8e86dba6_4ff2881f-e0a4-4f13-9d0e-e6020f0ac070'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:10 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1113003310-8e86dba6_4ff2881f-e0a4-4f13-9d0e-e6020f0ac070'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:11 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:13 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:17 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:17 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:17 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1113003310-8e86dba6_4ff2881f-e0a4-4f13-9d0e-e6020f0ac070 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43057.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44313.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45027
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:19 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113003310-8e86dba6_4ff2881f-e0a4-4f13-9d0e-e6020f0ac070: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.12 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/13 00:33:21 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1113003310-8e86dba6_4ff2881f-e0a4-4f13-9d0e-e6020f0ac070 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636763602.059522500","description":"Error received from peer ipv4:127.0.0.1:45027","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636763602.060986577","description":"Error received from peer ipv4:127.0.0.1:43057","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636763602.059522500","description":"Error received from peer ipv4:127.0.0.1:45027","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636763602.059533043","description":"Error received from peer ipv4:127.0.0.1:44313","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 46m 48s
214 actionable tasks: 166 executed, 44 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rmpueud7g7aok
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4508 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4508 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4508/ to view the results.
beam_PostCommit_Python37 - Build # 4507 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4507 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4507/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4506
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4506/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #15926 from [BEAM-13110][Playground] Playground
[noreply] [BEAM-13025] Disable deduplicating messages, as Dedupe is broken on
[noreply] [BEAM-13001] collect DoFn metrics for Combine (#15911)
------------------------------------------
[...truncated 51.14 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 174 warnings in 5866.84 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 44m 41s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/n4lwqvk6xjeiy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4505
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4505/display/redirect?page=changes>
Changes:
[Kyle Weaver] Update Beam website to release 2.34.0.
[Kyle Weaver] Prepare docs for 2.34.0 release.
[noreply] [BEAM-13133] Loosen partitioning requirement for sample (#15818)
[noreply] [BEAM-13228] fix data race in metrics.store (#15946)
[noreply] [BEAM-3293] Add binding cases for MultiMap side inputs (#15943)
[noreply] [BEAM-13222] Re-enable Spanner integration tests (#15948)
------------------------------------------
[...truncated 633.32 KB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 179039239
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 284376382
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 297916889
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 298163890
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 298294305
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 298406124
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 299015760
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636676353
nanos: 299166202
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.680572748184204 seconds.
INFO:root:Successfully completed job in 5.680572748184204 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:37333
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fcd9a170200> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fcd9a170290> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fcd9a1709e0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempi_u_uvlc/artifactsewlknff4' '--job-port' '36829' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:36829.
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:21 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46833'
WARNING:root:Waiting for grpc channel to be ready at localhost:36829.
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39565'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36829'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:22 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:36829.
WARNING:root:Waiting for grpc channel to be ready at localhost:36829.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_a1582484-9119-4857-8e97-d546bbe937ba.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_a1582484-9119-4857-8e97-d546bbe937ba.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_a1582484-9119-4857-8e97-d546bbe937ba.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:25 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_a1582484-9119-4857-8e97-d546bbe937ba.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:26 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1112001926-6ef7dbd6_6940b6e1-0d80-4dc7-aea8-0a8a59348dc2'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:26 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1112001926-6ef7dbd6_6940b6e1-0d80-4dc7-aea8-0a8a59348dc2'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:27 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:29 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:33 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:33 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:33 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1112001926-6ef7dbd6_6940b6e1-0d80-4dc7-aea8-0a8a59348dc2 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37995.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42321.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34561
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:35 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:35 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1112001926-6ef7dbd6_6940b6e1-0d80-4dc7-aea8-0a8a59348dc2: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/12 00:19:36 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1112001926-6ef7dbd6_6940b6e1-0d80-4dc7-aea8-0a8a59348dc2 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636676377.535511036","description":"Error received from peer ipv4:127.0.0.1:34561","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636676377.535511036","description":"Error received from peer ipv4:127.0.0.1:34561","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636676377.535542313","description":"Error received from peer ipv4:127.0.0.1:37995","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636676377.535521132","description":"Error received from peer ipv4:127.0.0.1:42321","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 50m 55s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3bdc55yjvzfzu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4504
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4504/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12550] Parallelizable kurtosis Implementation (#15909)
[noreply] [BEAM-13016] Remove avro-python3 dependency from Beam (#15900)
------------------------------------------
[...truncated 41.29 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 57 passed, 26 skipped, 169 warnings in 5747.90 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 225
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 45m 36s
214 actionable tasks: 153 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ubaizbmgd635w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4503
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4503/display/redirect?page=changes>
Changes:
[noreply] BEAM-13189 Python TextIO: add escapechar feature. (#15901)
------------------------------------------
[...truncated 794.37 KB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3222-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 111867189
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-13"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 225070953
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-13"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 241870164
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 242106437
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 242239952
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 242344379
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 244563341
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636632876
nanos: 244749307
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 12.401583671569824 seconds.
INFO:root:Successfully completed job in 12.401583671569824 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:35009
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd4594f9cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd4594f9d40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd4594f64d0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempy7rtueev/artifactsrw2pgg6j' '--job-port' '39693' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:42 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:34679'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40765'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39693'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_8180d834-5ab2-4360-8a06-1a55ee294871.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_8180d834-5ab2-4360-8a06-1a55ee294871.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_8180d834-5ab2-4360-8a06-1a55ee294871.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:43 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_8180d834-5ab2-4360-8a06-1a55ee294871.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:44 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1111121444-ee2da57d_6438f02e-9ae2-48fe-a319-ac49bdc232ae'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:44 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1111121444-ee2da57d_6438f02e-9ae2-48fe-a319-ac49bdc232ae'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:44 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:45 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:45 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:45 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1111121444-ee2da57d_6438f02e-9ae2-48fe-a319-ac49bdc232ae on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40967.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41175.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39365
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1111121444-ee2da57d_6438f02e-9ae2-48fe-a319-ac49bdc232ae: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/11 12:14:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1111121444-ee2da57d_6438f02e-9ae2-48fe-a319-ac49bdc232ae finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636632888.939673370","description":"Error received from peer ipv4:127.0.0.1:40967","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636632888.939658206","description":"Error received from peer ipv4:127.0.0.1:39365","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636632888.939658206","description":"Error received from peer ipv4:127.0.0.1:39365","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636632888.939673301","description":"Error received from peer ipv4:127.0.0.1:41175","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:direct:py37:postCommitIT
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 56s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4xtjzztbgkauk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4502
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4502/display/redirect?page=changes>
Changes:
[asottile] remove unused future / futures dependencies
[avilovpavel6] Fix environment_service_test.go
[noreply] [BEAM-3304] refactoring trigger package in GoSDK (#15933)
[noreply] [BEAM-13001] Hook to configure sampling period for DoFn metric
[noreply] [BEAM-3293] Add MultiMap side input decomposition (#15934)
[noreply] [BEAM-12651] Exclude packages from jacoco report (#15792)
[noreply] [BEAM-13222] Skip spanner integration tests (#15942)
------------------------------------------
[...truncated 295.98 KB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:root:Running (((((ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Impulse_15)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-FlatMap-lambda-at-core-py-3222-_16))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-DoOnce-Map-decode-_18))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-InitializeWrite_19))+(ref_PCollection_PCollection_10/Write))+(ref_PCollection_PCollection_11/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 171860456
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 353282213
}
message: "Renamed 1 shards in 0.18 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 385031461
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 385324478
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 385447025
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 385521650
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 385931730
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636611214
nanos: 386050224
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.802635669708252 seconds.
INFO:root:Successfully completed job in 10.802635669708252 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45719
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fb32c894c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fb32c894cb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fb32c891440> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempa7yxj9aq/artifactskll_u8fj' '--job-port' '54325' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:41 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:43711'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:41 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:44199'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:41 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:54325'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:41 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_1e808bae-0437-497b-aa9b-6c4c2ad4ffaa.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_1e808bae-0437-497b-aa9b-6c4c2ad4ffaa.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_1e808bae-0437-497b-aa9b-6c4c2ad4ffaa.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_1e808bae-0437-497b-aa9b-6c4c2ad4ffaa.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1111061342-d4bc3825_f48a3639-89d3-4d55-b786-109abda7cf89'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:42 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1111061342-d4bc3825_f48a3639-89d3-4d55-b786-109abda7cf89'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:43 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:43 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:44 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:44 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1111061342-d4bc3825_f48a3639-89d3-4d55-b786-109abda7cf89 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40347.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38691.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45775
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:45 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1111061342-d4bc3825_f48a3639-89d3-4d55-b786-109abda7cf89: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/11 06:13:46 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1111061342-d4bc3825_f48a3639-89d3-4d55-b786-109abda7cf89 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636611227.100935875","description":"Error received from peer ipv4:127.0.0.1:38691","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636611227.100944656","description":"Error received from peer ipv4:127.0.0.1:45775","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636611227.100944656","description":"Error received from peer ipv4:127.0.0.1:45775","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636611227.100935874","description":"Error received from peer ipv4:127.0.0.1:40347","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 44s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/uydau2fwt6qro
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4501 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4501 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4501/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4500
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4500/display/redirect>
Changes:
------------------------------------------
[...truncated 24.77 MB...]
INFO:root:Running (((((ref_AppliedPTransform_add-points-Impulse_3)+(ref_AppliedPTransform_add-points-FlatMap-lambda-at-core-py-3222-_4))+(ref_AppliedPTransform_add-points-Map-decode-_6))+(ref_AppliedPTransform_Map-get_julia_set_point_color-_7))+(ref_AppliedPTransform_x-coord-key_8))+(x coord/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:root:Running (((((x coord/Read)+(ref_AppliedPTransform_format_10))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WindowInto-WindowIntoFn-_20))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-WriteBundles_21))+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Pair_22))+(WriteToText/Write/WriteImpl/GroupByKey/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 581239461
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 744258642
}
message: "Renamed 1 shards in 0.16 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 756479740
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 756650447
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 756725788
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 756793498
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 757194757
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636568489
nanos: 757314443
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.190066576004028 seconds.
INFO:root:Successfully completed job in 6.190066576004028 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36507
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f1d736deb90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f1d736dec20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f1d736e23b0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempy953i5_s/artifactspwahxtkh' '--job-port' '56699' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:56699.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:38959'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:40999'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56699'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:37 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_fd6e1159-d415-4bcc-a90a-091ec6259a17.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_fd6e1159-d415-4bcc-a90a-091ec6259a17.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_fd6e1159-d415-4bcc-a90a-091ec6259a17.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:38 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_fd6e1159-d415-4bcc-a90a-091ec6259a17.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:39 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1110182139-d6383344_94c0e33f-95d8-4128-b04a-7e03b8147ad3'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:39 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1110182139-d6383344_94c0e33f-95d8-4128-b04a-7e03b8147ad3'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:39 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:41 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:44 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:44 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1110182139-d6383344_94c0e33f-95d8-4128-b04a-7e03b8147ad3 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41417.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38591.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37195
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1110182139-d6383344_94c0e33f-95d8-4128-b04a-7e03b8147ad3: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 18:21:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1110182139-d6383344_94c0e33f-95d8-4128-b04a-7e03b8147ad3 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636568508.612557504","description":"Error received from peer ipv4:127.0.0.1:37195","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636568508.612583831","description":"Error received from peer ipv4:127.0.0.1:41417","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636568508.612557504","description":"Error received from peer ipv4:127.0.0.1:37195","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636568508.612578612","description":"Error received from peer ipv4:127.0.0.1:38591","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 2s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ndjvch4l632c2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4499
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4499/display/redirect>
Changes:
------------------------------------------
[...truncated 48.02 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 177 warnings in 5796.96 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 41m 49s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/7vmtvzqgzj2zk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4498
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4498/display/redirect>
Changes:
------------------------------------------
[...truncated 31.96 MB...]
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:root:Running ((WriteToText/Write/WriteImpl/GroupByKey/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-Extract_24))+(ref_PCollection_PCollection_16/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:root:Running ((ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-PreFinalize_25))+(ref_PCollection_PCollection_17/Write)
INFO:apache_beam.runners.portability.fn_api_runner.fn_runner:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:Running (ref_PCollection_PCollection_10/Read)+(ref_AppliedPTransform_WriteToText-Write-WriteImpl-FinalizeWrite_26)
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 34689903
}
message: "Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1"
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 200890779
}
message: "Renamed 1 shards in 0.17 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 210403442
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 210547924
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 210620641
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 210688114
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 212119340
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636525071
nanos: 212267160
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.646186113357544 seconds.
INFO:root:Successfully completed job in 5.646186113357544 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:46441
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3c08602b90> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3c08602c20> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3c086063b0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempr8gqd1la/artifactsmentj5n5' '--job-port' '50103' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:50103.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:17:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:32911'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:17:58 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:43321'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:17:59 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50103'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:17:59 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:50103.
WARNING:root:Waiting for grpc channel to be ready at localhost:50103.
WARNING:root:Waiting for grpc channel to be ready at localhost:50103.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_f17df875-3c8b-49ee-b6a9-c616cba62b67.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_f17df875-3c8b-49ee-b6a9-c616cba62b67.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_f17df875-3c8b-49ee-b6a9-c616cba62b67.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:03 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_f17df875-3c8b-49ee-b6a9-c616cba62b67.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:04 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1110061804-d2f92735_c970c2e3-4a07-4a31-a08b-e3529e74e4be'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:04 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1110061804-d2f92735_c970c2e3-4a07-4a31-a08b-e3529e74e4be'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:05 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:06 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:09 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:09 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1110061804-d2f92735_c970c2e3-4a07-4a31-a08b-e3529e74e4be on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44597.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41549.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41757
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1110061804-d2f92735_c970c2e3-4a07-4a31-a08b-e3529e74e4be: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/10 06:18:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1110061804-d2f92735_c970c2e3-4a07-4a31-a08b-e3529e74e4be finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636525093.489409210","description":"Error received from peer ipv4:127.0.0.1:41757","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636525093.489409210","description":"Error received from peer ipv4:127.0.0.1:41757","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636525093.489416212","description":"Error received from peer ipv4:127.0.0.1:41549","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636525093.489435244","description":"Error received from peer ipv4:127.0.0.1:44597","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 50m 36s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/lgd3jzprzgvpg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4497 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4497 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4497/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4496
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4496/display/redirect?page=changes>
Changes:
[Andrew Pilloud] [BEAM-13056] Expose FieldAccess in DoFnSchemaInformation
[noreply] [BEAM-11097] Create hook to enable cross-bundle side input caching
[noreply] Merge pull request #15804 from [BEAM-13109][Playground] Add processing
------------------------------------------
[...truncated 48.10 MB...]
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:128
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:128: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
sql="select * from Users")
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:190
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:190: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_read_it_test.py:117
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_read_it_test.py>:117: FutureWarning: ReadFromSpanner is experimental. No backwards-compatibility guarantees.
columns=["UserId", "Key"])
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:171
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:171: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
database_id=self.TEST_DATABASE))
apache_beam/io/gcp/experimental/spannerio_write_it_test.py:135
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/experimental/spannerio_write_it_test.py>:135: FutureWarning: WriteToSpanner is experimental. No backwards-compatibility guarantees.
max_batch_size_bytes=250))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 1 failed, 62 passed, 21 skipped, 177 warnings in 6572.75 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 46s
214 actionable tasks: 157 executed, 53 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/u463jwcbe6cyk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Python37 - Build # 4495 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Python37 - Build # 4495 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Python37/4495/ to view the results.
Build failed in Jenkins: beam_PostCommit_Python37 #4494
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4494/display/redirect>
Changes:
------------------------------------------
[...truncated 48.16 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 170 warnings, 15 error in 6521.39 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 32s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/pptm324qitxhe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4493
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4493/display/redirect?page=changes>
Changes:
[aydar.zaynutdinov] [BEAM-13155][Playground]
[noreply] Merge pull request #15873 from [BEAM-13181] Remove Sharding from
[noreply] Merge pull request #15813 from [BEAM-13071] [Playground] Update run code
[Kyle Weaver] [BEAM-5172] Temporary ignore testSplit and testSizes tests waiting for a
------------------------------------------
[...truncated 48.24 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 180 warnings, 15 error in 5533.85 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 32s
214 actionable tasks: 153 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/axhnak7fd5zg4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4492
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4492/display/redirect?page=changes>
Changes:
[25622840+adude3141] [BEAM-13157] add regression test for hadoop configuration on
------------------------------------------
[...truncated 48.05 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 2 failed, 56 passed, 11 skipped, 176 warnings, 15 error in 5590.93 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 143
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 0s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ytpgacefsd3s2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4491
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4491/display/redirect>
Changes:
------------------------------------------
[...truncated 38.73 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 905283689
}
message: "Renamed 1 shards in 0.15 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 934769630
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 934989213
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 935072183
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 935149192
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 936298847
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636373914
nanos: 936441659
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.517492294311523 seconds.
INFO:root:Successfully completed job in 8.517492294311523 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33989
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f3e4b8800e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f3e4b880170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f3e4b8808c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempa0ot2qzh/artifactsnovynfy8' '--job-port' '50841' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:46371'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:38217'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:50841'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:39 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_e5790de4-5011-4d91-920f-74c259a1da00.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_e5790de4-5011-4d91-920f-74c259a1da00.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_e5790de4-5011-4d91-920f-74c259a1da00.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_e5790de4-5011-4d91-920f-74c259a1da00.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1108121840-174bcf97_2280632d-159d-4082-92d9-144a4cece1f0'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:40 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1108121840-174bcf97_2280632d-159d-4082-92d9-144a4cece1f0'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:41 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:41 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:42 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:42 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:42 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1108121840-174bcf97_2280632d-159d-4082-92d9-144a4cece1f0 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:35269.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35667.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37495
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1108121840-174bcf97_2280632d-159d-4082-92d9-144a4cece1f0: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/08 12:18:44 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1108121840-174bcf97_2280632d-159d-4082-92d9-144a4cece1f0 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636373925.158457274","description":"Error received from peer ipv4:127.0.0.1:37495","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636373925.158499115","description":"Error received from peer ipv4:127.0.0.1:35667","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636373925.158457274","description":"Error received from peer ipv4:127.0.0.1:37495","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636373925.158499047","description":"Error received from peer ipv4:127.0.0.1:35269","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 39s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/awuxu25roy2oc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4490
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4490/display/redirect>
Changes:
------------------------------------------
[...truncated 48.15 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 178 warnings, 15 error in 6427.84 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 16s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/owpvvnqk7iray
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4489
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4489/display/redirect>
Changes:
------------------------------------------
[...truncated 48.27 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 170 warnings, 15 error in 5590.81 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 17s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4vw5iq7ujcmoa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4488
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4488/display/redirect>
Changes:
------------------------------------------
[...truncated 822.37 KB...]
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 955495595
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 960185527
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 960331439
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 960407972
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 960473775
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 960994243
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636309482
nanos: 961114168
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 10.97054409980774 seconds.
INFO:root:Successfully completed job in 10.97054409980774 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36673
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f363951a0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f363951a170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f363951a8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempbv5h1pcg/artifacts7nlmx_2q' '--job-port' '42931' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:42931.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:32877'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42909'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:42931'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:42931.
WARNING:root:Waiting for grpc channel to be ready at localhost:42931.
WARNING:root:Waiting for grpc channel to be ready at localhost:42931.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_8ea67dc8-2db9-4783-ae42-0a5abe6e301f.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_8ea67dc8-2db9-4783-ae42-0a5abe6e301f.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:55 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_8ea67dc8-2db9-4783-ae42-0a5abe6e301f.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:56 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_8ea67dc8-2db9-4783-ae42-0a5abe6e301f.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:56 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1107182456-a93c5c60_46b6788b-bdf8-4710-bb14-1b729f87034f'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:56 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1107182456-a93c5c60_46b6788b-bdf8-4710-bb14-1b729f87034f'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:57 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:24:59 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:02 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:02 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:02 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1107182456-a93c5c60_46b6788b-bdf8-4710-bb14-1b729f87034f on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37253.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33853.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:45193
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:04 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107182456-a93c5c60_46b6788b-bdf8-4710-bb14-1b729f87034f: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:05 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 18:25:05 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107182456-a93c5c60_46b6788b-bdf8-4710-bb14-1b729f87034f finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636309506.042489529","description":"Error received from peer ipv4:127.0.0.1:45193","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636309506.042567079","description":"Error received from peer ipv4:127.0.0.1:37253","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636309506.042489529","description":"Error received from peer ipv4:127.0.0.1:45193","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636309506.042516175","description":"Error received from peer ipv4:127.0.0.1:33853","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 54m 47s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/evcwn5w6njdui
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4487
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4487/display/redirect>
Changes:
------------------------------------------
[...truncated 27.13 MB...]
timestamp {
seconds: 1636289372
nanos: 859606504
}
message: "Renamed 1 shards in 0.13 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-11"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 887414216
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 887634515
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 887731313
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 887816190
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 891565084
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636289372
nanos: 891776084
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 15.457299947738647 seconds.
INFO:root:Successfully completed job in 15.457299947738647 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:36451
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7fd2f3ced0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7fd2f3ced170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7fd2f3ced8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempvgeq5w5i/artifactsjp1r7lt5' '--job-port' '56337' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:50 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42607'
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:51 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39043'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:56337'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:52 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:56337.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_835c2a84-29b4-441c-8eb1-117e42eacbb6.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_835c2a84-29b4-441c-8eb1-117e42eacbb6.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_835c2a84-29b4-441c-8eb1-117e42eacbb6.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:57 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_835c2a84-29b4-441c-8eb1-117e42eacbb6.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:58 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1107124958-d3267f4a_20377d93-d3ea-492b-a6e2-9dc62b3f1d2b'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:49:59 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1107124958-d3267f4a_20377d93-d3ea-492b-a6e2-9dc62b3f1d2b'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:00 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:02 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:09 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:09 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:09 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1107124958-d3267f4a_20377d93-d3ea-492b-a6e2-9dc62b3f1d2b on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:39731.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:16 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:41731.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37807
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:16 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:17 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:18 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107124958-d3267f4a_20377d93-d3ea-492b-a6e2-9dc62b3f1d2b: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:20 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.15 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 12:50:20 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107124958-d3267f4a_20377d93-d3ea-492b-a6e2-9dc62b3f1d2b finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636289421.775137427","description":"Error received from peer ipv4:127.0.0.1:37807","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636289421.775137427","description":"Error received from peer ipv4:127.0.0.1:37807","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636289421.778114516","description":"Error received from peer ipv4:127.0.0.1:41731","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636289421.779293905","description":"Error received from peer ipv4:127.0.0.1:39731","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 21s
214 actionable tasks: 157 executed, 53 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/3xuseags75c2y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4486
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4486/display/redirect>
Changes:
------------------------------------------
[...truncated 32.51 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 983893871
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 989381551
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 989563941
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 989667415
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 989760398
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 991516351
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636265881
nanos: 991703033
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 8.55872631072998 seconds.
INFO:root:Successfully completed job in 8.55872631072998 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39863
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f75edc9c0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f75edc9c170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f75edc9c8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempse202g2n/artifactsz5w3sfxx' '--job-port' '45991' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:32847'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:37243'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:45991'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:07 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_c6c9458f-5838-403e-b82b-250f7f61d92a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_c6c9458f-5838-403e-b82b-250f7f61d92a.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_c6c9458f-5838-403e-b82b-250f7f61d92a.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:08 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_c6c9458f-5838-403e-b82b-250f7f61d92a.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:08 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1107061808-4723f4b6_7ff2f1f8-2c49-4252-8596-783a054bf51c'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:09 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1107061808-4723f4b6_7ff2f1f8-2c49-4252-8596-783a054bf51c'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:09 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:09 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:10 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:10 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:10 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1107061808-4723f4b6_7ff2f1f8-2c49-4252-8596-783a054bf51c on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44931.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40263.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:34807
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:11 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:11 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107061808-4723f4b6_7ff2f1f8-2c49-4252-8596-783a054bf51c: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 06:18:12 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107061808-4723f4b6_7ff2f1f8-2c49-4252-8596-783a054bf51c finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636265893.183905484","description":"Error received from peer ipv4:127.0.0.1:34807","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636265893.183905484","description":"Error received from peer ipv4:127.0.0.1:34807","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636265893.183912333","description":"Error received from peer ipv4:127.0.0.1:40263","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636265893.183918233","description":"Error received from peer ipv4:127.0.0.1:44931","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 49m 1s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bcntdqktytmzs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4485
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4485/display/redirect?page=changes>
Changes:
[relax] remove log statement
------------------------------------------
[...truncated 4.01 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 41951894
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 56932687
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 57088613
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 57200193
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 57283163
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 57771921
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636244361
nanos: 57882070
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.3958680629730225 seconds.
INFO:root:Successfully completed job in 5.3958680629730225 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39413
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f86c064d0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f86c064d170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f86c064d8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempmzpm2mj0/artifacts03d_v0q3' '--job-port' '46069' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:46069.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:36183'
WARNING:root:Waiting for grpc channel to be ready at localhost:46069.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:45751'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:46069'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:29 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:46069.
WARNING:root:Waiting for grpc channel to be ready at localhost:46069.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_46fb7ca9-21f8-4705-8c01-14056a70d5e2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_46fb7ca9-21f8-4705-8c01-14056a70d5e2.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:33 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_46fb7ca9-21f8-4705-8c01-14056a70d5e2.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:34 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_46fb7ca9-21f8-4705-8c01-14056a70d5e2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:34 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1107001934-fd43ad74_30f763cd-070b-45c2-91bf-b8110feb30cb'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:34 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1107001934-fd43ad74_30f763cd-070b-45c2-91bf-b8110feb30cb'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:35 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:37 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:40 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:40 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:40 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1107001934-fd43ad74_30f763cd-070b-45c2-91bf-b8110feb30cb on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44051.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44295.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43521
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:42 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:42 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107001934-fd43ad74_30f763cd-070b-45c2-91bf-b8110feb30cb: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/07 00:19:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1107001934-fd43ad74_30f763cd-070b-45c2-91bf-b8110feb30cb finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636244384.304237874","description":"Error received from peer ipv4:127.0.0.1:44295","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636244384.304207852","description":"Error received from peer ipv4:127.0.0.1:43521","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636244384.304314447","description":"Error received from peer ipv4:127.0.0.1:44051","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636244384.304207852","description":"Error received from peer ipv4:127.0.0.1:43521","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 45s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4oqi5hsxktsy2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4484
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4484/display/redirect>
Changes:
------------------------------------------
[...truncated 48.12 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 174 warnings, 15 error in 5702.37 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 39m 3s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/bd77nvtqqeamu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4483
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4483/display/redirect>
Changes:
------------------------------------------
[...truncated 48.24 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m== 1 failed, 57 passed, 11 skipped, 174 warnings, 15 error in 6477.56 seconds ==[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 12s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/iiswki7vumkns
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4482
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4482/display/redirect?page=changes>
Changes:
[joseinigo] [BEAM-13080] Fix number of default keys
[joseinigo] [BEAM-13080] Fix number of default keys
------------------------------------------
[...truncated 24.87 MB...]
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 842691421
}
message: "Renamed 1 shards in 0.10 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 857611417
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 857771635
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 857851743
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 857940673
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 858951568
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636179501
nanos: 859090328
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 4.7933549880981445 seconds.
INFO:root:Successfully completed job in 4.7933549880981445 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:33721
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f26f07620e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f26f0762170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f26f07628c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempml4u46p2/artifactsz2zswsj6' '--job-port' '55745' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:26 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:44403'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:26 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:34955'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:26 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:55745'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:26 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_14cbfdcd-33bc-420c-9a40-22baaf4f74de.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_14cbfdcd-33bc-420c-9a40-22baaf4f74de.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_14cbfdcd-33bc-420c-9a40-22baaf4f74de.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_14cbfdcd-33bc-420c-9a40-22baaf4f74de.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1106061827-ca53b738_d85562f6-64c6-4c34-af1b-384a80c8e120'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1106061827-ca53b738_d85562f6-64c6-4c34-af1b-384a80c8e120'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:27 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:28 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:28 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:28 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1106061827-ca53b738_d85562f6-64c6-4c34-af1b-384a80c8e120 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:38861.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:35447.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:44645
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:29 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:29 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1106061827-ca53b738_d85562f6-64c6-4c34-af1b-384a80c8e120: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/06 06:18:30 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1106061827-ca53b738_d85562f6-64c6-4c34-af1b-384a80c8e120 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636179511.300969485","description":"Error received from peer ipv4:127.0.0.1:44645","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636179511.300969485","description":"Error received from peer ipv4:127.0.0.1:44645","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636179511.300963377","description":"Error received from peer ipv4:127.0.0.1:35447","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636179511.300974786","description":"Error received from peer ipv4:127.0.0.1:38861","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 42s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
Publishing build scan failed due to network error 'java.net.SocketTimeoutException: Read timed out' (2 retries remaining)...
https://gradle.com/s/ert7dam4ov6hq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4481
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4481/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11217] Metrics Query filtering for DoFn metrics. (#15887)
[noreply] [Go SDK] Go SDK Exits Experimental (#15894)
------------------------------------------
[...truncated 26.21 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 58 passed, 11 skipped, 184 warnings, 15 error in 6483.95 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 52m 10s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ovxledy6bsgi2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4480
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4480/display/redirect?page=changes>
Changes:
[noreply] [adhoc] Speedup slow tests for AWS IO modules (#15899)
[noreply] [BEAM-12566] Implement set_axis for DataFrame and Series (#15773)
[noreply] [BEAM-13192] Fix buggy retry tests for AWS SnsIO (#15910)
------------------------------------------
[...truncated 190.89 KB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 473763227
}
message: "Renamed 1 shards in 0.11 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 486348390
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 486574172
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 486703395
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 486835956
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 488093376
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636137009
nanos: 488252401
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 5.6112682819366455 seconds.
INFO:root:Successfully completed job in 5.6112682819366455 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:43077
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7efd774550e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7efd77455170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7efd774558c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempejww5730/artifactswsemw5wt' '--job-port' '39371' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42121'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:39165'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:39371'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:14 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_2de04c5a-c5dc-488d-8fc4-88d17d38940d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_2de04c5a-c5dc-488d-8fc4-88d17d38940d.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_2de04c5a-c5dc-488d-8fc4-88d17d38940d.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_2de04c5a-c5dc-488d-8fc4-88d17d38940d.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1105183017-5e4379e_23bf5dda-9b5c-44a7-81a9-55110afdc79c'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:17 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1105183017-5e4379e_23bf5dda-9b5c-44a7-81a9-55110afdc79c'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:18 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:18 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:19 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:19 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:19 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1105183017-5e4379e_23bf5dda-9b5c-44a7-81a9-55110afdc79c on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:44101.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:33689.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:37095
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:21 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105183017-5e4379e_23bf5dda-9b5c-44a7-81a9-55110afdc79c: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:22 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 18:30:22 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105183017-5e4379e_23bf5dda-9b5c-44a7-81a9-55110afdc79c finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636137022.960771561","description":"Error received from peer ipv4:127.0.0.1:33689","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636137022.960771163","description":"Error received from peer ipv4:127.0.0.1:44101","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636137022.960742338","description":"Error received from peer ipv4:127.0.0.1:37095","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636137022.960742338","description":"Error received from peer ipv4:127.0.0.1:37095","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 10s
214 actionable tasks: 153 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/vz6prku4o7pyy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4479
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4479/display/redirect>
Changes:
------------------------------------------
[...truncated 1.12 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 800864696
}
message: "Renamed 1 shards in 0.18 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 835120677
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 835308313
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 835401296
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 835515737
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 837034702
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636115238
nanos: 837268114
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 33.66959738731384 seconds.
INFO:root:Successfully completed job in 33.66959738731384 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:39945
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f45cfa9a0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f45cfa9a170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f45cfa9a8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temphfq7blcc/artifacts_c21b_3q' '--job-port' '44327' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:44327.
WARNING:root:Waiting for grpc channel to be ready at localhost:44327.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:33 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:42979'
WARNING:root:Waiting for grpc channel to be ready at localhost:44327.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:41229'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44327'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:34 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:44327.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_47f1abf5-4275-4903-903c-2e48cfaac6e2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_47f1abf5-4275-4903-903c-2e48cfaac6e2.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_47f1abf5-4275-4903-903c-2e48cfaac6e2.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:36 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_47f1abf5-4275-4903-903c-2e48cfaac6e2.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:37 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1105122737-dacc849_f5f5f6af-07f2-4165-a744-7416cb604c12'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:37 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1105122737-dacc849_f5f5f6af-07f2-4165-a744-7416cb604c12'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:38 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:40 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:43 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:43 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:43 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1105122737-dacc849_f5f5f6af-07f2-4165-a744-7416cb604c12 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37239.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:46 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:43169.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43741
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:46 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:47 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:48 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105122737-dacc849_f5f5f6af-07f2-4165-a744-7416cb604c12: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.13 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 12:27:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105122737-dacc849_f5f5f6af-07f2-4165-a744-7416cb604c12 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636115269.986319433","description":"Error received from peer ipv4:127.0.0.1:43741","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636115269.986319433","description":"Error received from peer ipv4:127.0.0.1:43741","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636115269.986326707","description":"Error received from peer ipv4:127.0.0.1:43169","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636115269.986353052","description":"Error received from peer ipv4:127.0.0.1:37239","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 38m 10s
214 actionable tasks: 150 executed, 60 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fwnkqwe5mwe4q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4478
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4478/display/redirect>
Changes:
------------------------------------------
[...truncated 32.28 MB...]
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 206568717
}
message: "Renamed 1 shards in 0.14 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 240433931
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 240591287
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 240672588
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 240770101
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 241775989
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636093228
nanos: 241889238
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 6.023723602294922 seconds.
INFO:root:Successfully completed job in 6.023723602294922 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:34747
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7f4c59c3d0e0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f4c59c3d170> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f4c59c3d8c0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempxku4mtqi/artifactswawq1u76' '--job-port' '36471' '--artifact-port' '0' '--expansion-port' '0']
WARNING:root:Waiting for grpc channel to be ready at localhost:36471.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:39395'
WARNING:root:Waiting for grpc channel to be ready at localhost:36471.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:36875'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:36471'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:36 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:root:Waiting for grpc channel to be ready at localhost:36471.
WARNING:root:Waiting for grpc channel to be ready at localhost:36471.
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_3f0cd355-e938-4571-88b7-8de4a3543022.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_3f0cd355-e938-4571-88b7-8de4a3543022.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_3f0cd355-e938-4571-88b7-8de4a3543022.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:41 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_3f0cd355-e938-4571-88b7-8de4a3543022.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:41 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1105062041-3f2839dd_a70a3f62-3fd3-4ea8-8220-fc77c15af125'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:42 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1105062041-3f2839dd_a70a3f62-3fd3-4ea8-8220-fc77c15af125'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:42 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:44 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:47 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:47 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1105062041-3f2839dd_a70a3f62-3fd3-4ea8-8220-fc77c15af125 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:37103.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:44039.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46787
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:49 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:50 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105062041-3f2839dd_a70a3f62-3fd3-4ea8-8220-fc77c15af125: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:51 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.10 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/05 06:20:51 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1105062041-3f2839dd_a70a3f62-3fd3-4ea8-8220-fc77c15af125 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636093251.734329234","description":"Error received from peer ipv4:127.0.0.1:46787","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636093251.734329234","description":"Error received from peer ipv4:127.0.0.1:46787","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636093251.734356641","description":"Error received from peer ipv4:127.0.0.1:37103","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636093251.734346711","description":"Error received from peer ipv4:127.0.0.1:44039","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 40m 20s
214 actionable tasks: 152 executed, 58 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/jsivafhxlcwus
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4477
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4477/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-3811] Code clean-up of the CancelleableQueue to not throw
[noreply] [BEAM-13081] Fixes a compatible issue of decoding null-value bitmap
------------------------------------------
[...truncated 48.12 MB...]
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
apache_beam/io/gcp/bigquery.py:2112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
apache_beam/io/gcp/bigquery.py:2398
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2398: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
apache_beam/io/gcp/bigquery.py:2400
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2400: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
apache_beam/io/gcp/bigquery.py:2431
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2431: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/dataframe/flight_delays.py:45
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
apache_beam/dataframe/io.py:572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/dataframe/io.py>:572: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
apache_beam/io/fileio.py:550
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
apache_beam/io/gcp/bigquery.py:2102
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2102: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
apache_beam/io/gcp/bigquery_file_loads.py:1112
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1112: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
apache_beam/io/gcp/bigquery_file_loads.py:1114
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1114: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/io/gcp/tests/utils.py:100
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/avro/schema.py>:1251: DeprecationWarning: `Parse` is deprecated in avro 1.9.2. Please use `parse` (lowercase) instead.
DeprecationWarning)
apache_beam/io/gcp/bigquery_test.py:1124
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_test.py>:1124: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
streaming = self.test_pipeline.options.view_as(StandardOptions).streaming
apache_beam/ml/gcp/cloud_dlp_it_test.py:77
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:77: FutureWarning: MaskDetectedDetails is experimental.
inspection_config=INSPECT_CONFIG))
apache_beam/io/gcp/bigquery_read_it_test.py:167
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:167: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
apache_beam/io/gcp/big_query_query_to_table_pipeline.py:84
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py>:84: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
kms_key=kms_key))
apache_beam/runners/dataflow/ptransform_overrides.py:323
apache_beam/runners/dataflow/ptransform_overrides.py:323
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/dataflow/ptransform_overrides.py>:323: BeamDeprecationWarning: BigQuerySink is deprecated since 2.11.0. Use WriteToBigQuery instead.
kms_key=self.kms_key))
apache_beam/ml/gcp/cloud_dlp_it_test.py:87
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/ml/gcp/cloud_dlp_it_test.py>:87: FutureWarning: InspectForDetails is experimental.
| beam.ParDo(extract_inspection_results).with_outputs(
apache_beam/io/gcp/bigquery_read_it_test.py:556
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:556: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_read_it_test.py:670
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:670: FutureWarning: ReadAllFromBigQuery is experimental.
| beam.io.ReadAllFromBigQuery())
apache_beam/io/gcp/bigquery.py:2572
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2572: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/io/gcp/bigquery.py:2573
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2573: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
project = pcoll.pipeline.options.view_as(GoogleCloudOptions).project
apache_beam/io/gcp/bigquery.py:2586
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2586: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
options=pcoll.pipeline.options,
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/pytest_postCommitIT-df-py37.xml> -
[31m[1m======= 58 passed, 11 skipped, 174 warnings, 15 error in 5907.68 seconds =======[0m
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 45m 27s
214 actionable tasks: 164 executed, 46 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/rtevf3otvhnqm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #4476
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python37/4476/display/redirect?page=changes>
Changes:
[heejong] [BEAM-13021] Deduplicate Python artifact not only by hash but also by
[heejong] add test
[heejong] rename variables
[mmack] [BEAM-11440] Add integration test for KinesisIO using Localstack on
[mmack] Bump localstack container version for more robust version with respect
[mmack] Bump localstack container version for DynamoDBIOTest to version used in
[mmack] Apply format
[mmack] Apply format
[mmack] Use vendored guava deps
[heejong] rearrange comparison order
[Luke Cwik] [BEAM-13164] Address most of a race condition between instantiation and
[Robert Bradshaw] Defer filesToStage construction until after full jar resolution.
[heejong] s/pathes/paths/
[noreply] Merge pull request #15835: [BEAM-11205] Google Libraries BOM 24.0.0 and
[noreply] Merge pull request #15810: [BEAM-2791] Support low-latency StorageApi
------------------------------------------
[...truncated 14.08 MB...]
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:303"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636050398
nanos: 997136592
}
message: "Renamed 1 shards in 0.12 seconds."
instruction_id: "bundle_6"
transform_id: "WriteToText/Write/WriteImpl/FinalizeWrite"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py:348"
thread: "Thread-14"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 24501800
}
message: "No more requests from control plane"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:244"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 24714469
}
message: "SDK Harness waiting for in-flight requests to complete"
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:245"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 24808883
}
message: "Closing all cached grpc data channels."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/data_plane.py:782"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 24878740
}
message: "Closing all cached gRPC state handlers."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:826"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 25423049
}
message: "Done consuming work."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py:257"
thread: "MainThread"
INFO:root:severity: INFO
timestamp {
seconds: 1636050399
nanos: 25554180
}
message: "Python sdk harness exiting."
log_location: "/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py:154"
thread: "MainThread"
INFO:apache_beam.runners.portability.local_job_service:Successfully completed job in 7.553454875946045 seconds.
INFO:root:Successfully completed job in 7.553454875946045 seconds.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
> Task :sdks:python:test-suites:portable:py37:portableWordCountSparkRunnerBatch
INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:40599
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.35.0.dev
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function pack_combiners at 0x7ff98592acb0> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff98592ad40> ====================
INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff9859264d0> ====================
INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.35.0-SNAPSHOT.jar'> '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-tempocui_tuw/artifactsbjud8lcc' '--job-port' '44971' '--artifact-port' '0' '--expansion-port' '0']
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: ArtifactStagingService started on localhost:37941'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Java ExpansionService started on localhost:42007'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: JobService started on localhost:44971'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:44 INFO org.apache.beam.runners.jobsubmission.JobServerDriver: Job server now running, terminate with Ctrl+C'
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['--parallelism=2']
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Staging artifacts for job_3e8e5863-beab-4424-a4a7-0befbee9f214.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Resolving artifacts for job_3e8e5863-beab-4424-a4a7-0befbee9f214.ref_Environment_default_environment_1.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Getting 1 artifacts for job_3e8e5863-beab-4424-a4a7-0befbee9f214.null.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService: Artifacts fully staged for job_3e8e5863-beab-4424-a4a7-0befbee9f214.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.spark.SparkJobInvoker: Invoking job BeamApp-jenkins-1104182645-77413bb9_b89946cf-d6a4-4086-8ea0-2453a9a4b609'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.jobsubmission.JobInvocation: Starting job invocation BeamApp-jenkins-1104182645-77413bb9_b89946cf-d6a4-4086-8ea0-2453a9a4b609'
INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
with Pipeline() as p:
p.apply(..)
This ensures that the pipeline finishes before this program exits.
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:45 INFO org.apache.beam.runners.spark.translation.SparkContextFactory: Creating a brand new Spark Context.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:46 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:47 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator:'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:47 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: MetricQueryResults()'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:47 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Running job BeamApp-jenkins-1104182645-77413bb9_b89946cf-d6a4-4086-8ea0-2453a9a4b609 on Spark master local[4]'
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:46099.
INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2'
INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:40803.
INFO:apache_beam.runners.worker.sdk_worker:State channel established.
INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:35831
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected.'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-3'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-4'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-5'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-7'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-6'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-8'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:48 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-9'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-10'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-11'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-12'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-13'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-14'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-15'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1104182645-77413bb9_b89946cf-d6a4-4086-8ea0-2453a9a4b609: Pipeline translated successfully. Computing outputs'
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-16'
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 4 (skipped: 0), batches: 4, num_threads: 4
INFO:apache_beam.io.filebasedsink:Renamed 4 shards in 0.11 seconds.
INFO:apache_beam.utils.subprocess_server:b'21/11/04 18:26:49 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-1104182645-77413bb9_b89946cf-d6a4-4086-8ea0-2453a9a4b609 finished.'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
Exception in thread run_worker_1-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 234, in run
for work_request in self._control_stub.Control(get_responses()):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Connection reset by peer"
debug_error_string = "{"created":"@1636050410.075814643","description":"Error received from peer ipv4:127.0.0.1:46099","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Connection reset by peer","grpc_status":14}"
>
Exception in thread read_state:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/sdk_worker.py",> line 957, in pull_responses
for response in responses:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636050410.075801180","description":"Error received from peer ipv4:127.0.0.1:40803","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
ERROR:apache_beam.runners.worker.data_plane:Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636050410.075776932","description":"Error received from peer ipv4:127.0.0.1:35831","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1636050410.075776932","description":"Error received from peer ipv4:127.0.0.1:35831","file":"src/core/lib/surface/call.cc","file_line":1069,"grpc_message":"Socket closed","grpc_status":14}"
>
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 53
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 120
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 37m 42s
214 actionable tasks: 164 executed, 46 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/uqjmrxde4r756
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org