You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/12/01 13:53:57 UTC
Build failed in Jenkins: beam_PostCommit_Python39 #2599
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2599/display/redirect>
Changes:
------------------------------------------
[...truncated 12.31 MB...]
[gw4] [33mSKIPPED[0m apache_beam/examples/inference/tfx_bsl/tfx_bsl_inference_it_test.py::TFXRunInferenceTests::test_tfx_run_inference_mobilenetv2
apache_beam/ml/gcp/visionml_test_it.py::VisionMlTestIT::test_text_detection_with_language_hint
[gw2] [32mPASSED[0m apache_beam/io/gcp/healthcare/dicomio_integration_test.py::DICOMIoIntegrationTest::test_dicom_store_instance_from_gcs
apache_beam/ml/gcp/naturallanguageml_test_it.py::NaturalLanguageMlTestIT::test_analyzing_syntax
[gw2] [33mSKIPPED[0m apache_beam/ml/gcp/naturallanguageml_test_it.py::NaturalLanguageMlTestIT::test_analyzing_syntax
apache_beam/ml/inference/base_test.py::RunInferenceBaseTest::test_run_inference_with_side_inputin_streaming
[gw2] [33mSKIPPED[0m apache_beam/ml/inference/base_test.py::RunInferenceBaseTest::test_run_inference_with_side_inputin_streaming
apache_beam/transforms/periodicsequence_it_test.py::PeriodicSequenceIT::test_periodicsequence_outputs_valid_watermarks_it
[gw2] [33mSKIPPED[0m apache_beam/transforms/periodicsequence_it_test.py::PeriodicSequenceIT::test_periodicsequence_outputs_valid_watermarks_it
[gw3] [32mPASSED[0m apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/io/parquetio_it_test.py::TestParquetIT::test_parquetio_it
[gw0] [32mPASSED[0m apache_beam/examples/complete/game/user_score_it_test.py::UserScoreIT::test_user_score_it
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
[gw6] [32mPASSED[0m apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/cookbook/datastore_wordcount_it_test.py::DatastoreWordCountIT::test_datastore_wordcount_it
[gw7] [32mPASSED[0m apache_beam/io/gcp/bigquery_test.py::BigQueryFileLoadsIntegrationTests::test_avro_file_load
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_native_datetime
apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_query
[gw4] [32mPASSED[0m apache_beam/ml/gcp/visionml_test_it.py::VisionMlTestIT::test_text_detection_with_language_hint
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_it
[gw4] [33mSKIPPED[0m apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_it
apache_beam/transforms/external_it_test.py::ExternalTransformIT::test_job_python_from_python_it
[gw5] [32mPASSED[0m apache_beam/examples/dataframe/taxiride_it_test.py::TaxirideIT::test_enrich
apache_beam/examples/streaming_wordcount_it_test.py::StreamingWordCountIT::test_streaming_wordcount_it
[gw3] [32mPASSED[0m apache_beam/io/parquetio_it_test.py::TestParquetIT::test_parquetio_it
apache_beam/io/gcp/datastore/v1new/datastore_write_it_test.py::DatastoreWriteIT::test_datastore_write_limit
[gw0] [32mPASSED[0m apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/ml/gcp/videointelligenceml_test_it.py::VideoIntelligenceMlTestIT::test_label_detection_with_video_context
[gw0] [33mSKIPPED[0m apache_beam/ml/gcp/videointelligenceml_test_it.py::VideoIntelligenceMlTestIT::test_label_detection_with_video_context
[gw6] [32mPASSED[0m apache_beam/examples/cookbook/datastore_wordcount_it_test.py::DatastoreWordCountIT::test_datastore_wordcount_it
apache_beam/io/fileio_test.py::MatchIntegrationTest::test_transform_on_gcs
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_query
apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_query_and_filters
[gw4] [32mPASSED[0m apache_beam/transforms/external_it_test.py::ExternalTransformIT::test_job_python_from_python_it
[gw6] [32mPASSED[0m apache_beam/io/fileio_test.py::MatchIntegrationTest::test_transform_on_gcs
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_query_and_filters
apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_row_restriction
[gw7] [32mPASSED[0m apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
apache_beam/ml/inference/onnx_inference_it_test.py::OnnxInference::test_onnx_run_inference_roberta_sentiment_classification
[gw7] [33mSKIPPED[0m apache_beam/ml/inference/onnx_inference_it_test.py::OnnxInference::test_onnx_run_inference_roberta_sentiment_classification
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_row_restriction
apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_very_selective_filters
[gw5] [32mPASSED[0m apache_beam/examples/streaming_wordcount_it_test.py::StreamingWordCountIT::test_streaming_wordcount_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadUsingStorageApiTests::test_iobase_source_with_very_selective_filters
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_iobase_source
[gw5] [32mPASSED[0m apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_iobase_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
[gw3] [32mPASSED[0m apache_beam/io/gcp/datastore/v1new/datastore_write_it_test.py::DatastoreWriteIT::test_datastore_write_limit
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadAllBQTests::test_read_queries
apache_beam/io/gcp/bigquery_read_it_test.py::ReadInteractiveRunnerTests::test_read_in_interactive_runner
[gw1] [32mPASSED[0m apache_beam/io/gcp/bigquery_read_it_test.py::ReadInteractiveRunnerTests::test_read_in_interactive_runner
=================================== FAILURES ===================================
[31m[1m_________________________ TaxirideIT.test_aggregation __________________________[0m
[gw5] linux -- Python 3.9.10 <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/build/gradleenv/-1734967050/bin/python3.9>
self = <apache_beam.examples.dataframe.taxiride_it_test.TaxirideIT testMethod=test_aggregation>
@pytest.mark.it_postcommit
def test_aggregation(self):
taxiride.run_aggregation_pipeline(
self.test_pipeline,
'gs://apache-beam-samples/nyc_taxi/2018/*.csv',
self.output_path)
# Verify
expected = pd.read_csv(
os.path.join(
os.path.dirname(__file__),
'data',
'taxiride_2018_aggregation_truth.csv'),
comment='#')
expected = expected.sort_values('DOLocationID').reset_index(drop=True)
def read_csv(path):
with FileSystems.open(path) as fp:
return pd.read_csv(fp)
result = pd.concat(
read_csv(metadata.path) for metadata in FileSystems.match(
[f'{self.output_path}*'])[0].metadata_list)
result = result.sort_values('DOLocationID').reset_index(drop=True)
> pd.testing.assert_frame_equal(expected, result)
[1m[31mapache_beam/examples/dataframe/taxiride_it_test.py[0m:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[1m[31mpandas/_libs/testing.pyx[0m:52: in pandas._libs.testing.assert_almost_equal
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
[1m[31mE AssertionError: DataFrame.iloc[:, 1] (column name="passenger_count") are different[0m
[1m[31mE [0m
[1m[31mE DataFrame.iloc[:, 1] (column name="passenger_count") values are different (95.83333 %)[0m
[1m[31mE [index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, ...][0m
[1m[31mE [left]: [322785, 126, 11462, 765196, 1129, 3233, 690775, 2745, 12449, 71098, 12153, 128193, 1554566, 121699, 15656, 24503, 216383, 24966, 10986, 15848, 21407, 31214, 4207, 543676, 309075, 38391, 1863, 41136, 16975, 574, 8262, 15028, 471728, 18156, 27688, 130194, 204445, 13124, 37766, 181880, 1082726, 724974, 2138199, 879, 476163, 2665, 16868, 4565446, 275157, 1918576, 22594, 133087, 16560, 43805, 21054, 44920, 8374, 3600, 1124, 11858, 258716, 68486, 17464, 13871, 308587, 253356, 26034, 3649382, 72461, 69129, 30635, 29204, 9531, 1290532, 1944536, 48657, 13536, 17909, 3793408, 266865, 11246, 140149, 63732, 1367, 25466, 10757, 1332524, 675278, 137834, 2395749, 32698, 70710, 29590, 12862, 192248, 2807, 268138, 20011, 89, 2551666, ...][0m
[1m[31mE [right]: [324609, 126, 11561, 767142, 1132, 3242, 693105, 2745, 12492, 71442, 12187, 129187, 1553915, 122015, 15707, 24597, 217078, 25063, 11004, 15927, 21459, 31234, 4207, 544171, 309134, 38323, 1863, 41409, 17018, 581, 8318, 15064, 471387, 18134, 27852, 130830, 205683, 13159, 37929, 182224, 1084447, 727157, 2140590, 881, 477647, 2675, 16937, 4574206, 275266, 1921643, 22683, 133178, 16617, 43852, 21087, 45154, 8471, 3609, 1128, 11861, 259615, 68621, 17505, 13907, 308662, 253662, 26107, 3648222, 72628, 69420, 30727, 29349, 9546, 1293391, 1947669, 48759, 13569, 17974, 3796068, 267714, 11300, 140963, 64081, 1377, 25583, 10786, 1331117, 676312, 138295, 2394029, 32774, 71059, 29873, 12915, 192905, 2809, 268305, 20065, 89, 2553916, ...][0m
[1m[31mpandas/_libs/testing.pyx[0m:167: AssertionError
------------------------------ Captured log call -------------------------------
[32mINFO [0m root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]]:140709409308096]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140709409252352]}, partitioning=Arbitrary, ops=[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]], outputs={PlaceholderExpression[placeholder_DataFrame_140709409252352], ComputedExpression[pre_combine_sum_Series_140709409254032]}]
[32mINFO [0m root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280]> for Stage[inputs={ComputedExpression[pre_combine_sum_Series_140709409254032]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_Series_140709409253696]], outputs={ComputedExpression[post_combine_sum_Series_140709409253696]}]
[32mINFO [0m apache_beam.io.fileio:fileio.py:611 Added temporary directory gs://temp-storage-for-end-to-end-tests/temp-it/.temp44db3d46-659c-4fc2-b699-2b30235d20df
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying Beam SDK "<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/build/apache_beam-2.53.0.dev0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"> to staging location.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline has additional dependencies to be installed in SDK worker container, consider using the SDK container image pre-building workflow to avoid repetitive installations. Learn more on https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:313 Using provided Python SDK container image: gcr.io/cloud-dataflow/v1beta3/beam_python3.9_sdk:beam-master-20231116
[32mINFO [0m root:environments.py:320 Python SDK container image set to "gcr.io/cloud-dataflow/v1beta3/beam_python3.9_sdk:beam-master-20231116" for Docker environment
[32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function pack_combiners at 0x7ff9a7934ee0> ====================
[32mINFO [0m apache_beam.runners.portability.fn_api_runner.translations:translations.py:712 ==================== <function sort_stages at 0x7ff9a7935700> ====================
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:673 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1201124314-345882-hrjx92wd.1701434594.346043/apache_beam-2.53.0.dev0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:683 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1201124314-345882-hrjx92wd.1701434594.346043/apache_beam-2.53.0.dev0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl in 0 seconds.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:673 Starting GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1201124314-345882-hrjx92wd.1701434594.346043/pipeline.pb...
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:683 Completed GCS upload to gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-1201124314-345882-hrjx92wd.1701434594.346043/pipeline.pb in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:338 Unknown pipeline options received: --sleep_secs=20,--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test. Ignore if flags are used for internal purposes.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:338 Unknown pipeline options received: --sleep_secs=20,--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test. Ignore if flags are used for internal purposes.
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:854 Create job: <Job
clientRequestId: '20231201124314346943-8510'
createTime: '2023-12-01T12:43:15.531470Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2023-12-01_04_43_15-11258339740913282460'
location: 'us-central1'
name: 'beamapp-jenkins-1201124314-345882-hrjx92wd'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2023-12-01T12:43:15.531470Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:856 Created job with id: [2023-12-01_04_43_15-11258339740913282460]
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:857 Submitted job: 2023-12-01_04_43_15-11258339740913282460
[32mINFO [0m apache_beam.runners.dataflow.internal.apiclient:apiclient.py:858 To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-12-01_04_43_15-11258339740913282460?project=apache-beam-testing
[32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58 Console log:
[32mINFO [0m apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59 https://console.cloud.google.com/dataflow/jobs/us-central1/2023-12-01_04_43_15-11258339740913282460?project=apache-beam-testing
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-12-01_04_43_15-11258339740913282460 is in state JOB_STATE_RUNNING
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:18.395Z: JOB_MESSAGE_BASIC: Worker configuration: e2-standard-2 in us-central1-b.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:21.592Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:21.656Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:22.227Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:30.852Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/Create/Impulse+_ReadFromPandas/Create/FlatMap(<lambda at core.py:3774>)+_ReadFromPandas/Create/Map(decode)+_ReadFromPandas/MatchAll/ParDo(_MatchAllFn)+_ReadFromPandas/Map(<lambda at io.py:299>)+_ReadFromPandas/Reshuffle/AddRandomKeys+_ReadFromPandas/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Reify+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:43:43.239Z: JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:17.734Z: JOB_MESSAGE_BASIC: All workers have finished the startup processes and began to receive work requests.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.086Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/Create/Impulse+_ReadFromPandas/Create/FlatMap(<lambda at core.py:3774>)+_ReadFromPandas/Create/Map(decode)+_ReadFromPandas/MatchAll/ParDo(_MatchAllFn)+_ReadFromPandas/Map(<lambda at io.py:299>)+_ReadFromPandas/Reshuffle/AddRandomKeys+_ReadFromPandas/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Reify+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.140Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.158Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/Map(<lambda at io.py:295>)/View-python_side_input0
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.181Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.203Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/Map(<lambda at io.py:295>)/View-python_side_input0
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:19.281Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/DoOnce/Impulse+_ReadFromPandas/DoOnce/FlatMap(<lambda at core.py:3774>)+_ReadFromPandas/DoOnce/Map(decode)+_ReadFromPandas/Map(<lambda at io.py:295>)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:21.081Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/DoOnce/Impulse+_ReadFromPandas/DoOnce/FlatMap(<lambda at core.py:3774>)+_ReadFromPandas/DoOnce/Map(decode)+_ReadFromPandas/Map(<lambda at io.py:295>)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:21.160Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/ParDo(_ReadFromPandasDoFn)/View-python_side_input0
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:21.206Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/ParDo(_ReadFromPandasDoFn)/View-python_side_input0
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:21.272Z: JOB_MESSAGE_BASIC: Executing operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Read+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+_ReadFromPandas/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+_ReadFromPandas/Reshuffle/RemoveRandomKeys+_ReadFromPandas/ReadMatches/ParDo(_ReadMatchesFn)+ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/PairWithRestriction+ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/SplitWithSizing
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:23.471Z: JOB_MESSAGE_BASIC: Finished operation _ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/Read+_ReadFromPandas/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+_ReadFromPandas/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+_ReadFromPandas/Reshuffle/RemoveRandomKeys+_ReadFromPandas/ReadMatches/ParDo(_ReadMatchesFn)+ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/PairWithRestriction+ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/SplitWithSizing
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:23.554Z: JOB_MESSAGE_BASIC: Executing operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:23.930Z: JOB_MESSAGE_BASIC: Finished operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:46:24.007Z: JOB_MESSAGE_BASIC: Executing operation ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/ProcessElementAndRestrictionWithSizing+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]]:140709409308096/Map(<lambda at transforms.py:245>)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]]:140709409308096/FlatMap(evaluate)/FlatMap(evaluate)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/ParDo(_PreBatch)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/FlatMap(partition_fn)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/Filter(<lambda at transforms.py:231>)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/Tag[0]+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/Flatten/OutputIdentity+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:43.895Z: JOB_MESSAGE_BASIC: Finished operation ref_AppliedPTransform__ReadFromPandas-ParDo-_ReadFromPandasDoFn-_26/ProcessElementAndRestrictionWithSizing+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]]:140709409308096/Map(<lambda at transforms.py:245>)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[set_index_DataFrame_140709410657856], ComputedExpression[pre_combine_sum_Series_140709409254032]]:140709409308096/FlatMap(evaluate)/FlatMap(evaluate)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/ParDo(_PreBatch)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/FlatMap(partition_fn)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/Partition_Index_pre_combine_sum_Series_140709409254032/Filter(<lambda at transforms.py:231>)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/Tag[0]+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/Flatten/OutputIdentity+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:43.932Z: JOB_MESSAGE_BASIC: Executing operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.273Z: JOB_MESSAGE_BASIC: Finished operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.319Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.481Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.558Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.717Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Create
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:44.797Z: JOB_MESSAGE_BASIC: Executing operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/MapTuple(collect_values)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/RestoreTags+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/ParDo(_ReBatch)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/FlatMap(evaluate)/FlatMap(evaluate)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_WriteUnshardedRecordsFn)/ParDo(_WriteUnshardedRecordsFn)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/Map(<lambda at fileio.py:640>)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Write+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_AppendShardedDestination)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:45.829Z: JOB_MESSAGE_BASIC: Finished operation ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/CoGroupByKeyImpl/MapTuple(collect_values)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/CoGroupByKey/RestoreTags+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/ParDo(_ReBatch)+ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/[ComputedExpression[post_combine_sum_Series_140709409253696]]:140709409255280/FlatMap(evaluate)/FlatMap(evaluate)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_WriteUnshardedRecordsFn)/ParDo(_WriteUnshardedRecordsFn)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/Map(<lambda at fileio.py:640>)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Write+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_AppendShardedDestination)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:45.875Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:45.919Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:45.959Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_WriteShardedRecordsFn)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/Map(<lambda at fileio.py:640>)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:46.275Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_WriteShardedRecordsFn)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/Map(<lambda at fileio.py:640>)+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Write
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:46.321Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:46.364Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Close
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:46.407Z: JOB_MESSAGE_BASIC: Executing operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Read+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_MoveTempFilesIntoFinalDestinationFn)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:47.044Z: JOB_MESSAGE_BASIC: Finished operation WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/GroupTempFilesByDestination/Read+WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/taxiride_it-122b4734-f3e3-424c-b03c-527d9a8abecf/output.csv/WriteToFiles/ParDo(_MoveTempFilesIntoFinalDestinationFn)
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:49:47.202Z: JOB_MESSAGE_BASIC: Stopping worker pool...
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:200 2023-12-01T12:51:51.232Z: JOB_MESSAGE_BASIC: Worker pool stopped.
[32mINFO [0m apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:150 Job 2023-12-01_04_43_15-11258339740913282460 is in state JOB_STATE_DONE
[32mINFO [0m apache_beam.io.gcp.gcsio:gcsio.py:486 Finished listing 1 files in 0.13379716873168945 seconds.
[32mINFO [0m apache_beam.io.gcp.gcsio:gcsio.py:486 Finished listing 1 files in 0.06843781471252441 seconds.
[33m=============================== warnings summary ===============================[0m
apache_beam/io/gcp/bigquery_read_it_test.py::ReadTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:170: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_file_loads
apache_beam/io/gcp/bigquery_test.py::PubSubBigQueryIT::test_streaming_inserts
apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_it
apache_beam/examples/complete/game/game_stats_it_test.py::GameStatsIT::test_game_stats_it
apache_beam/examples/complete/game/leader_board_it_test.py::LeaderBoardIT::test_leader_board_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
dataset_ref = client.dataset(unique_dataset_name, project=project)
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:47: FutureWarning: The default value of numeric_only in DataFrame.mean is deprecated. In a future version, it will default to False. In addition, specifying 'numeric_only=None' is deprecated. Select only valid columns or specify the value of numeric_only to silence this warning.
return airline_df[at_top_airports].mean()
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
table_ref = client.dataset(dataset_id).table(table_id)
apache_beam/io/gcp/bigquery_read_it_test.py::ReadNewTypesTests::test_native_source
<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py>:706: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
beam.io.BigQuerySource(query=self.query, use_standard_sql=True)))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/pytest_postCommitIT-df-py39.xml> -
[36m[1m=========================== short test summary info ============================[0m
[31mFAILED[0m apache_beam/examples/dataframe/taxiride_it_test.py::[1mTaxirideIT::test_aggregation[0m - AssertionError: DataFrame.iloc[:, 1] (column name="passenger_count") are different
DataFrame.iloc[:, 1] (column name="passenger_count") values are different (95.83333 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, ...]
[left]: [322785, 126, 11462, 765196, 1129, 3233, 690775, 2745, 12449, 71098, 12153, 128193, 1554566, 121699, 15656, 24503, 216383, 24966, 10986, 15848, 21407, 31214, 4207, 543676, 309075, 38391, 1863, 41136, 16975, 574, 8262, 15028, 471728, 18156, 27688, 130194, 204445, 13124, 37766, 181880, 1082726, 724974, 2138199, 879, 476163, 2665, 16868, 4565446, 275157, 1918576, 22594, 133087, 16560, 43805, 21054, 44920, 8374, 3600, 1124, 11858, 258716, 68486, 17464, 13871, 308587, 253356, 26034, 3649382, 72461, 69129, 30635, 29204, 9531, 1290532, 1944536, 48657, 13536, 17909, 3793408, 266865, 11246, 140149, 63732, 1367, 25466, 10757, 1332524, 675278, 137834, 2395749, 32698, 70710, 29590, 12862, 192248, 2807, 268138, 20011, 89, 2551666, ...]
[right]: [324609, 126, 11561, 767142, 1132, 3242, 693105, 2745, 12492, 71442, 12187, 129187, 1553915, 122015, 15707, 24597, 217078, 25063, 11004, 15927, 21459, 31234, 4207, 544171, 309134, 38323, 1863, 41409, 17018, 581, 8318, 15064, 471387, 18134, 27852, 130830, 205683, 13159, 37929, 182224, 1084447, 727157, 2140590, 881, 477647, 2675, 16937, 4574206, 275266, 1921643, 22683, 133178, 16617, 43852, 21087, 45154, 8471, 3609, 1128, 11861, 259615, 68621, 17505, 13907, 308662, 253662, 26107, 3648222, 72628, 69420, 30727, 29349, 9546, 1293391, 1947669, 48759, 13569, 17974, 3796068, 267714, 11300, 140963, 64081, 1377, 25583, 10786, 1331117, 676312, 138295, 2394029, 32774, 71059, 29873, 12915, 192905, 2809, 268305, 20065, 89, 2553916, ...]
[31m====== [31m[1m1 failed[0m, [32m82 passed[0m, [33m50 skipped[0m, [33m9 warnings[0m[31m in 5720.72s (1:35:20)[0m[31m =======[0m
> Task :sdks:python:test-suites:dataflow:py39:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python39/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 139
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py39:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 1h 42m 32s
222 actionable tasks: 157 executed, 61 from cache, 4 up-to-date
Publishing build scan...
https://ge.apache.org/s/jrt4vawc7rhcm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python39 #2600
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python39/2600/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org