You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/02/09 18:08:36 UTC
Build failed in Jenkins: beam_PostCommit_Python_Examples_Spark #20
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/20/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK
[noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776)
[noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for
------------------------------------------
[...truncated 3.16 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:sharded_key:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:progress_reporting:v0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:harness_monitoring_infos:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:worker_status:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:combinefn:packed_python:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.37.0.dev"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:transform:to_string:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'22/02/09 18:08:27 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'22/02/09 18:08:27 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0209180805-7827896b_71c1851e-3472-42b6-926d-220c68b5a109 finished.'
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:576 Job state changed to DONE
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.03394460678100586 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.04135465621948242 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.030356645584106445 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 12 files in 0.03612780570983887 seconds.
PASSED [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_139880314567440], ComputedExpression[set_index_DataFrame_139880314568336], ComputedExpression[pre_combine_sum_DataFrame_139882074142672]]:139881651795024]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_139882074919440]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_139880314567440], ComputedExpression[set_index_DataFrame_139880314568336], ComputedExpression[pre_combine_sum_DataFrame_139882074142672]], outputs={ComputedExpression[pre_combine_sum_DataFrame_139882074142672], PlaceholderExpression[placeholder_DataFrame_139882074919440]}]
INFO root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_139882072097104]]:139882075349584]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_139882074142672]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_139882072097104]], outputs={ComputedExpression[post_combine_sum_DataFrame_139882072097104]}]
INFO apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.temp16092784-763d-4fae-bb6c-ba0b5578e0b1
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7f38c16bf950> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7f38c16bfa70> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f38c16bff80> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7f38c16be050> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7f38c16be200> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7f38c16be290> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7f38c16be3b0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7f38c16be440> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7f38c16be4d0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7f38c16be560> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f38c16be7a0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7f38c16be710> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7f38c16be830> ====================
INFO apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f38d504d710> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmpinctmc7r.result/[ComputedExpression[post_combine_sum_DataFrame_139882072097104]]:139882075349584/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmpinctmc7r.result/[ComputedExpression[post_combine_sum_DataFrame_139882072097104]]:139882075349584/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmpinctmc7r.result/[ComputedExpression[post_combine_sum_DataFrame_139882072097104]]:139882075349584/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpinctmc7r-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpinctmc7r-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_139882072097104-with-indexes-ParDo-_Unbatch_46))+(WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpinctmc7r-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpinctmc7r-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/Flatten/Write/1)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpinctmc7r-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmpinctmc7r.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpinctmc7r-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.temp16092784-763d-4fae-bb6c-ba0b5578e0b1/8237360927517950451_99d850dc-c7e1-4ae8-8528-89694de60bda to dir: /tmp as tmpinctmc7r.result-00000-of-00001. Res: FileResult(file_name='/tmp/.temp16092784-763d-4fae-bb6c-ba0b5578e0b1/8237360927517950451_99d850dc-c7e1-4ae8-8528-89694de60bda', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/pytest_postCommitExamples-spark-py37.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 39 warnings in 291.02 seconds =====
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644430067.569271512","description":"Error received from peer ipv6:[::1]:44509","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644430104.676333419","description":"Error received from peer ipv6:[::1]:39535","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644430104.817925414","description":"Error received from peer ipv6:[::1]:45253","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644430105.026488156","description":"Error received from peer ipv6:[::1]:43655","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 248
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkExamples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 6m 32s
79 actionable tasks: 50 executed, 27 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/o4coiapax6hgi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python_Examples_Spark #22
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/22/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python_Examples_Spark #21
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/21/display/redirect?page=changes>
Changes:
[david.prieto.rivera] Missing contribution
[noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation
[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.
[noreply] [release-2.36.0][website] Fix github release notes script, header for
[noreply] Use shell to run python for setupVirtualenv (#16796)
[Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script.
[noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to
[noreply] [BEAM-13776][Playground] (#16731)
------------------------------------------
[...truncated 3.17 MB...]
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03903937339782715 seconds.
INFO apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.0432584285736084 seconds.
INFO apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'22/02/10 00:22:53 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'payload: "\\n\\021\\n\\017localhost:37603"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:bytes:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:string_utf8:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:kv:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:bool:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:244 No more requests from control plane
ERROR apache_beam.runners.worker.data_plane:data_plane.py:641 Failed to read inputs in the data plane.
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644452573.798698010","description":"Error received from peer ipv6:[::1]:32955","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:varint:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:245 SDK Harness waiting for in-flight requests to complete
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:double:v1"'
INFO apache_beam.runners.worker.data_plane:data_plane.py:782 Closing all cached grpc data channels.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:iterable:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:826 Closing all cached gRPC state handlers.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:timer:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:interval_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:length_prefix:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:257 Done consuming work.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:global_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:param_windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:state_backed_iterable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:custom_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:row:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:coder:sharded_key:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:progress_reporting:v0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:harness_monitoring_infos:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:protocol:worker_status:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:combinefn:packed_python:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.37.0.dev"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'capabilities: "beam:transform:to_string:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'22/02/10 00:22:53 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'22/02/10 00:22:53 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job BeamApp-jenkins-0210002231-6ac583c5_b9b7e459-44ba-488a-bab3-9b01ba4d1160 finished.'
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:576 Job state changed to DONE
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.03881978988647461 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.0488133430480957 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 4 files in 0.03709292411804199 seconds.
INFO apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 12 files in 0.04759550094604492 seconds.
PASSED [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
-------------------------------- live log call ---------------------------------
INFO root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140663561685904], ComputedExpression[set_index_DataFrame_140663561683472], ComputedExpression[pre_combine_sum_DataFrame_140665108525392]]:140663557980688]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140665890329744]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140663561685904], ComputedExpression[set_index_DataFrame_140663561683472], ComputedExpression[pre_combine_sum_DataFrame_140665108525392]], outputs={PlaceholderExpression[placeholder_DataFrame_140665890329744], ComputedExpression[pre_combine_sum_DataFrame_140665108525392]}]
INFO root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140663561649872]]:140663561787664]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140665108525392]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140663561649872]], outputs={ComputedExpression[post_combine_sum_DataFrame_140663561649872]}]
INFO apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.tempdb1ec988-867d-4277-8771-5d57cb76f07d
WARNING root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
INFO root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.7_sdk:2.37.0.dev
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7fef2ab4c7a0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7fef2ab4c8c0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fef2ab4cdd0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7fef2ab4ce60> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7fef2ab44050> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7fef2ab440e0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7fef2ab44200> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7fef2ab44290> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7fef2ab44320> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7fef2ab443b0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fef2ab445f0> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7fef2ab44560> ====================
INFO apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7fef2ab44680> ====================
INFO apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7fef262b01d0> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmp12ydq6_n.result/[ComputedExpression[post_combine_sum_DataFrame_140663561649872]]:140663561787664/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmp12ydq6_n.result/[ComputedExpression[post_combine_sum_DataFrame_140663561649872]]:140663561787664/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmp12ydq6_n.result/[ComputedExpression[post_combine_sum_DataFrame_140663561649872]]:140663561787664/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp12ydq6_n-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp12ydq6_n-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140663561649872-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp12ydq6_n-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp12ydq6_n-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/Flatten/Write/1)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp12ydq6_n-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmp12ydq6_n.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp12ydq6_n-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.tempdb1ec988-867d-4277-8771-5d57cb76f07d/1356248840311852570_990cebc3-1dbe-4f6f-b204-6207c1b68045 to dir: /tmp as tmp12ydq6_n.result-00000-of-00001. Res: FileResult(file_name='/tmp/.tempdb1ec988-867d-4277-8771-5d57cb76f07d/1356248840311852570_990cebc3-1dbe-4f6f-b204-6207c1b68045', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
is_streaming_pipeline = p.options.view_as(StandardOptions).streaming
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
experiments = p.options.view_as(DebugOptions).experiments or []
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = p.options.view_as(GoogleCloudOptions).temp_location
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
| _PassThroughThenCleanup(files_to_remove_pcoll))
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
self.table_reference.projectId = pcoll.pipeline.options.view_as(
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError. Select only valid columns before calling the reduction.
return airline_df[at_top_airports].mean()
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
sink=lambda _: _WriteToPandasFileSink(
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
p.options.view_as(GoogleCloudOptions).temp_location or
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/pytest_postCommitExamples-spark-py37.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 39 warnings in 265.30 seconds =====
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644452530.059189296","description":"Error received from peer ipv6:[::1]:44433","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
Exception in thread read_grpc_client_inputs:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 651, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/apache_beam/runners/worker/data_plane.py",> line 634, in _read_inputs
for elements in elements_iterator:
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 426, in __next__
return self._next()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/build/gradleenv/2022703441/lib/python3.7/site-packages/grpc/_channel.py",> line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "{"created":"@1644452570.680984687","description":"Error received from peer ipv6:[::1]:39431","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"Multiplexer hanging up","grpc_status":1}"
>
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/ws/src/sdks/python/test-suites/portable/common.gradle'> line: 248
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py38:sparkExamples'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 7m 21s
79 actionable tasks: 52 executed, 25 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/nbj56nmqnvfds
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org