You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/02/08 06:06:36 UTC

Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #18

See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/18/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13848] Update numpy intersphinx link (#16767)

[noreply] [release-23.6.0] Fix JIRA link for 2.36 blog (#16771)

[noreply] [BEAM-13647] Use role for Go worker binary. (#16729)


------------------------------------------
[...truncated 801.41 KB...]
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 39 warnings in 207.98 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03597092628479004 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((((((((((((((((ref_PCollection_PCollection_6_split/Read)+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-PassThrough-ParDo-PassThrough-_16))+(ref_AppliedPTransform_assign-timestamp_23))+(ref_PCollection_PCollection_9/Write))+(ref_AppliedPTransform_set-schema-Map-lambda-at-core-py-2836-_25))+(ref_AppliedPTransform_daily-windows_26))+(ref_AppliedPTransform_BatchElements-daily-BatchElements-ParDo-_WindowAwareBatchingDoFn-_29))+(ref_AppliedPTransform_BatchElements-daily-Map-lambda-at-schemas-py-140-_30))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__33))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__35))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__38))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__39))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__40))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__43))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/[ComputedExpression[apply_DataFrame_140335771854608]]:140335770790096/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__44))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/[ComputedExpression[apply_DataFrame_140335771854608]]:140335770790096/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/[ComputedExpression[apply_DataFrame_140335771854608]]:140335770790096/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.09335780143737793 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/8486159956238836654_d2b85e66-b009-45b4-9109-f207780377d3 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/8486159956238836654_d2b85e66-b009-45b4-9109-f207780377d3', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.050984859466552734 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/4273148420993145938_afb53d9d-5880-499f-a453-f1db9506c6ad to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/4273148420993145938_afb53d9d-5880-499f-a453-f1db9506c6ad', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03827500343322754 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/7974773739052588114_51ce6916-a21f-4646-ad2d-1fea7b31d158 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a822aa6c-854a-401c-8b3c-d93d57dacf8b as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp27535ae8-2c72-4151-8e91-a41d9bee80cb/7974773739052588114_51ce6916-a21f-4646-ad2d-1fea7b31d158', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03306317329406738 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03150582313537598 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03536200523376465 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.06905293464660645 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.03960871696472168 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140335741107224], ComputedExpression[set_index_DataFrame_140335743405472], ComputedExpression[pre_combine_sum_DataFrame_140335743402336]]:140338340591208]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140335741901344]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140335741107224], ComputedExpression[set_index_DataFrame_140335743405472], ComputedExpression[pre_combine_sum_DataFrame_140335743402336]], outputs={PlaceholderExpression[placeholder_DataFrame_140335741901344], ComputedExpression[pre_combine_sum_DataFrame_140335743402336]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140335741104424]]:140335740995176]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140335743402336]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140335741104424]], outputs={ComputedExpression[post_combine_sum_DataFrame_140335741104424]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.tempccae8ffd-562b-439f-ab61-32fa98dcbeb0
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7fa2886e52f0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7fa2886e5400> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fa2886e58c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7fa2886e5950> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7fa2886e5ae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7fa2886e5b70> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7fa2886e5c80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7fa2886e5d08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7fa2886e5d90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7fa2886e5e18> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fa2886f50d0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7fa2886f5048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7fa2886f5158> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7fa284360390> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmp343cef5q.result/[ComputedExpression[post_combine_sum_DataFrame_140335741104424]]:140335740995176/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmp343cef5q.result/[ComputedExpression[post_combine_sum_DataFrame_140335741104424]]:140335740995176/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmp343cef5q.result/[ComputedExpression[post_combine_sum_DataFrame_140335741104424]]:140335740995176/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp343cef5q-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp343cef5q-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140335741104424-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp343cef5q-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp343cef5q-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp343cef5q-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmp343cef5q.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp343cef5q-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.tempccae8ffd-562b-439f-ab61-32fa98dcbeb0/5562587804829775427_197ec6cc-528c-4acb-9f9b-359b6fa22ed4 to dir: /tmp as tmp343cef5q.result-00000-of-00001. Res: FileResult(file_name='/tmp/.tempccae8ffd-562b-439f-ab61-32fa98dcbeb0/5562587804829775427_197ec6cc-528c-4acb-9f9b-359b6fa22ed4', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 38 warnings in 211.93 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 5m 40s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/p5ciluua3pxjc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Jenkins build is back to normal : beam_PostCommit_Python_Examples_Direct #26

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/26/display/redirect?page=changes>


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #25

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/25/display/redirect?page=changes>

Changes:

[david.prieto.rivera] Missing contribution

[noreply] Merge pull request #15848 from [BEAM-13835] An any-type implementation

[Valentyn Tymofieiev] [BEAM-12920] Assume that bare generators types define simple generators.

[noreply] [release-2.36.0][website] Fix github release notes script, header for

[noreply] Use shell to run python for setupVirtualenv (#16796)

[Daniel Oliveira] [BEAM-13830] Properly shut down Debezium expansion service in IT script.

[noreply] Merge pull request #16659 from [BEAM-13774][Playground] Add user to

[noreply] [BEAM-13776][Playground] (#16731)


------------------------------------------
[...truncated 801.22 KB...]
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 39 warnings in 185.55 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:557 Started BigQuery job: <JobReference
                                                                  jobId: 'beam_bq_job_EXPORT_BQ_EXPORT_JOB_bbf10e78-e_1644452254_543'
                                                                  location: 'US'
                                                                  projectId: 'apache-beam-testing'>
                                                                  bq show -j --format=prettyjson --project_id=apache-beam-testing beam_bq_job_EXPORT_BQ_EXPORT_JOB_bbf10e78-e_1644452254_543
INFO     root:bigquery_tools.py:625 Job status: RUNNING
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.04726052284240723 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((((((((((((((((ref_PCollection_PCollection_6_split/Read)+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-PassThrough-ParDo-PassThrough-_16))+(ref_AppliedPTransform_assign-timestamp_23))+(ref_PCollection_PCollection_9/Write))+(ref_AppliedPTransform_set-schema-Map-lambda-at-core-py-2836-_25))+(ref_AppliedPTransform_daily-windows_26))+(ref_AppliedPTransform_BatchElements-daily-BatchElements-ParDo-_WindowAwareBatchingDoFn-_29))+(ref_AppliedPTransform_BatchElements-daily-Map-lambda-at-schemas-py-140-_30))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__33))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__35))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__38))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__39))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__40))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__43))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/[ComputedExpression[apply_DataFrame_140315601137736]]:140315602071904/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__44))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/[ComputedExpression[apply_DataFrame_140315601137736]]:140315602071904/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/[ComputedExpression[apply_DataFrame_140315601137736]]:140315602071904/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/7974773739052588114_6c718c2f-4864-44b6-aa4f-a67f05f25524 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/7974773739052588114_6c718c2f-4864-44b6-aa4f-a67f05f25524', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03866839408874512 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/8486159956238836654_c412816a-2b34-4930-8dc5-27c18ff2870d to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/8486159956238836654_c412816a-2b34-4930-8dc5-27c18ff2870d', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.04186058044433594 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/4273148420993145938_ccc91a38-f608-4467-a596-20adeab0e59a to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-152b51ef-c338-45bd-a90f-b47c84164dad as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp855fe0ae-f1d8-4859-ae5c-6a961bb583d7/4273148420993145938_ccc91a38-f608-4467-a596-20adeab0e59a', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03099679946899414 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.09845590591430664 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03463912010192871 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.07831072807312012 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03490090370178223 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.04360628128051758 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140316614100080], ComputedExpression[set_index_DataFrame_140315597801232], ComputedExpression[pre_combine_sum_DataFrame_140315597324240]]:140316616010664]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140315597772952]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140316614100080], ComputedExpression[set_index_DataFrame_140315597801232], ComputedExpression[pre_combine_sum_DataFrame_140315597324240]], outputs={ComputedExpression[pre_combine_sum_DataFrame_140315597324240], PlaceholderExpression[placeholder_DataFrame_140315597772952]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140315597801064]]:140315597772672]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140315597324240]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140315597801064]], outputs={ComputedExpression[post_combine_sum_DataFrame_140315597801064]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.temp87424769-fef3-4b76-8e05-d199e85c1deb
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7f9dc9bd4268> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7f9dc9bd4378> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f9dc9bd4840> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7f9dc9bd48c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7f9dc9bd4a60> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7f9dc9bd4ae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7f9dc9bd4bf8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7f9dc9bd4c80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7f9dc9bd4d08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7f9dc9bd4d90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f9dc9bd1048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7f9dc9bd4f28> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7f9dc9bd10d0> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f9dc54df160> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmpmgidok1p.result/[ComputedExpression[post_combine_sum_DataFrame_140315597801064]]:140315597772672/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmpmgidok1p.result/[ComputedExpression[post_combine_sum_DataFrame_140315597801064]]:140315597772672/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmpmgidok1p.result/[ComputedExpression[post_combine_sum_DataFrame_140315597801064]]:140315597772672/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpmgidok1p-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpmgidok1p-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140315597801064-with-indexes-ParDo-_Unbatch_46))+(WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpmgidok1p-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpmgidok1p-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpmgidok1p-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmpmgidok1p.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpmgidok1p-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.temp87424769-fef3-4b76-8e05-d199e85c1deb/5562587925071239802_f67e5baf-08a8-44c8-a087-0fd4484eaccb to dir: /tmp as tmpmgidok1p.result-00000-of-00001. Res: FileResult(file_name='/tmp/.temp87424769-fef3-4b76-8e05-d199e85c1deb/5562587925071239802_f67e5baf-08a8-44c8-a087-0fd4484eaccb', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 38 warnings in 191.08 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 47s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/cv5rhqvpgxns4

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #24

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/24/display/redirect?page=changes>

Changes:

[noreply] [BEAM-13803] Add support for native iterable side inputs to the Go SDK

[noreply] [BEAM-11095] Better error handling for illegal emit functions (#16776)

[noreply] Merge pull request #16613 from Supporting JdbcIO driver in classpath for


------------------------------------------
[...truncated 802.95 KB...]
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 39 warnings in 227.96 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/[ComputedExpression[apply_DataFrame_140594893127352]]:140594899775216/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/7974773739052588114_1bdd58ac-c496-4236-8bed-bfc646f19709 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/7974773739052588114_1bdd58ac-c496-4236-8bed-bfc646f19709', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.05883431434631348 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/8486159956238836654_16b68f9e-b917-4852-9761-3b5daa8c95cb to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/8486159956238836654_16b68f9e-b917-4852-9761-3b5daa8c95cb', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03809022903442383 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/4273148420993145938_ba1cf0b7-61ce-4af8-80a5-cbea1c5aeb9b to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-6564975f-0119-463a-bf6a-6a4ed991529c as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp2036db13-5e0c-4a05-a40b-be20cd8b0d7b/4273148420993145938_ba1cf0b7-61ce-4af8-80a5-cbea1c5aeb9b', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.04351806640625 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.10687255859375 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.08140921592712402 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.04337286949157715 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.044689178466796875 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.046457529067993164 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140594893384336], ComputedExpression[set_index_DataFrame_140595774958336], ComputedExpression[pre_combine_sum_DataFrame_140594887934920]]:140595774958728]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140594896708888]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140594893384336], ComputedExpression[set_index_DataFrame_140595774958336], ComputedExpression[pre_combine_sum_DataFrame_140594887934920]], outputs={PlaceholderExpression[placeholder_DataFrame_140594896708888], ComputedExpression[pre_combine_sum_DataFrame_140594887934920]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140594664453400]]:140594893381760]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140594887934920]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140594664453400]], outputs={ComputedExpression[post_combine_sum_DataFrame_140594664453400]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.tempc91be98f-3c64-47da-8dc8-fe6972cb3612
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7fded10fb378> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7fded10fb488> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fded10fb950> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7fded10fb9d8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7fded10fbb70> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7fded10fbbf8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7fded10fbd08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7fded10fbd90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7fded10fbe18> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7fded10fbea0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fded1102158> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7fded11020d0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7fded11021e0> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7fdecd0f2278> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmpw_uu1n3t.result/[ComputedExpression[post_combine_sum_DataFrame_140594664453400]]:140594893381760/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmpw_uu1n3t.result/[ComputedExpression[post_combine_sum_DataFrame_140594664453400]]:140594893381760/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmpw_uu1n3t.result/[ComputedExpression[post_combine_sum_DataFrame_140594664453400]]:140594893381760/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpw_uu1n3t-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpw_uu1n3t-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140594664453400-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpw_uu1n3t-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpw_uu1n3t-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpw_uu1n3t-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmpw_uu1n3t.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpw_uu1n3t-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.tempc91be98f-3c64-47da-8dc8-fe6972cb3612/5562587875516492877_269e9bd6-d3a0-4d74-bd9e-5cbaab032191 to dir: /tmp as tmpw_uu1n3t.result-00000-of-00001. Res: FileResult(file_name='/tmp/.tempc91be98f-3c64-47da-8dc8-fe6972cb3612/5562587875516492877_269e9bd6-d3a0-4d74-bd9e-5cbaab032191', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5184 deselected, 38 warnings in 226.26 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 5m 9s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/pkhdyuhvggl34

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #23

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/23/display/redirect?page=changes>

Changes:

[mmack] [adhoc] Remove remaining usage of Powermock from aws2.


------------------------------------------
[...truncated 800.44 KB...]
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 39 warnings in 214.60 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.05221414566040039 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((((((((((((((((ref_PCollection_PCollection_6_split/Read)+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-PassThrough-ParDo-PassThrough-_16))+(ref_PCollection_PCollection_9/Write))+(ref_AppliedPTransform_assign-timestamp_23))+(ref_AppliedPTransform_set-schema-Map-lambda-at-core-py-2836-_25))+(ref_AppliedPTransform_daily-windows_26))+(ref_AppliedPTransform_BatchElements-daily-BatchElements-ParDo-_WindowAwareBatchingDoFn-_29))+(ref_AppliedPTransform_BatchElements-daily-Map-lambda-at-schemas-py-140-_30))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__33))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__35))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__38))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__39))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__40))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__43))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/[ComputedExpression[apply_DataFrame_139777346667184]]:139777346804312/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__44))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/[ComputedExpression[apply_DataFrame_139777346667184]]:139777346804312/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/[ComputedExpression[apply_DataFrame_139777346667184]]:139777346804312/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/4273148420993145938_ca3c177a-3097-40e8-927f-4e4ba6350a6d to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/4273148420993145938_ca3c177a-3097-40e8-927f-4e4ba6350a6d', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.02756953239440918 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/8486159956238836654_684e7600-1bee-4ea8-9514-58e74ce9a3f1 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/8486159956238836654_684e7600-1bee-4ea8-9514-58e74ce9a3f1', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03510332107543945 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/7974773739052588114_7fadb4a2-b3ac-405c-a7fd-2e1aae7f86d3 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-f561a661-db28-4c59-844f-1427dd314b2f as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp1e5fdce9-d3ae-4cfb-859e-344fd3d942b7/7974773739052588114_7fadb4a2-b3ac-405c-a7fd-2e1aae7f86d3', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.05088353157043457 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03428387641906738 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03571915626525879 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.029890060424804688 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03267931938171387 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.03241229057312012 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_139776455988224], ComputedExpression[set_index_DataFrame_139776469690072], ComputedExpression[pre_combine_sum_DataFrame_139776463320400]]:139777349702264]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_139777343876904]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_139776455988224], ComputedExpression[set_index_DataFrame_139776469690072], ComputedExpression[pre_combine_sum_DataFrame_139776463320400]], outputs={ComputedExpression[pre_combine_sum_DataFrame_139776463320400], PlaceholderExpression[placeholder_DataFrame_139777343876904]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_139776463322304]]:139776442069848]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_139776463320400]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_139776463322304]], outputs={ComputedExpression[post_combine_sum_DataFrame_139776463322304]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.temp4856d69c-d0c2-452a-a5ba-81d0d558e823
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7f2042c9f268> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7f2042c9f378> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f2042c9f840> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7f2042c9f8c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7f2042c9fa60> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7f2042c9fae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7f2042c9fbf8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7f2042c9fc80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7f2042c9fd08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7f2042c9fd90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f2042ca4048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7f2042c9ff28> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7f2042ca40d0> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f203e8e14a8> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmperu4d7vj.result/[ComputedExpression[post_combine_sum_DataFrame_139776463322304]]:139776442069848/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmperu4d7vj.result/[ComputedExpression[post_combine_sum_DataFrame_139776463322304]]:139776442069848/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmperu4d7vj.result/[ComputedExpression[post_combine_sum_DataFrame_139776463322304]]:139776442069848/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmperu4d7vj-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmperu4d7vj-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_139776463322304-with-indexes-ParDo-_Unbatch_46))+(WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmperu4d7vj-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmperu4d7vj-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmperu4d7vj-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmperu4d7vj.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmperu4d7vj-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.temp4856d69c-d0c2-452a-a5ba-81d0d558e823/5562588595976933802_cf3433da-dd85-4af9-929f-eae452aae145 to dir: /tmp as tmperu4d7vj.result-00000-of-00001. Res: FileResult(file_name='/tmp/.temp4856d69c-d0c2-452a-a5ba-81d0d558e823/5562588595976933802_cf3433da-dd85-4af9-929f-eae452aae145', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 38 warnings in 213.42 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 6m
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/mxm3aqtid3ozy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #22

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/22/display/redirect?page=changes>

Changes:

[Kyle Weaver] Update Dataflow Python dev container images.

[Kiley Sok] Add java 17 to changes

[Daniel Oliveira] [BEAM-13732] Switch x-lang BigQueryIO expansion service to GCP one.

[noreply] [BEAM-13858] Fix broken github action on :sdks:go:examples:wordCount

[Kiley Sok] add jira for runner v2

[noreply] [BEAM-13732] Go SDK BigQuery IO wrapper. Initial implementation.

[noreply] [BEAM-13732] Add example for Go BigQuery IO wrapper. (#16786)

[noreply] Update CHANGES.md with Go SDK milestones. (#16787)

[noreply] [BEAM-13193] Allow BeamFnDataOutboundObserver to flush elements.


------------------------------------------
[...truncated 806.38 KB...]
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 39 warnings in 251.28 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.05558133125305176 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.tempf2c88350-105c-4ef3-9f6a-76700cfc19e2/8486159956238836654_211d5f46-6c3c-4067-b5dc-8c750463adef to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a1b5f36d-7608-44bd-af52-f1c255f2094b as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.tempf2c88350-105c-4ef3-9f6a-76700cfc19e2/8486159956238836654_211d5f46-6c3c-4067-b5dc-8c750463adef', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03581643104553223 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.tempf2c88350-105c-4ef3-9f6a-76700cfc19e2/7974773739052588114_c316854d-3fa2-4f9d-897d-a7abca4de68c to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-a1b5f36d-7608-44bd-af52-f1c255f2094b as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.tempf2c88350-105c-4ef3-9f6a-76700cfc19e2/7974773739052588114_c316854d-3fa2-4f9d-897d-a7abca4de68c', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.05642557144165039 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03861641883850098 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03632020950317383 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.043350934982299805 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.04353952407836914 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.0572962760925293 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140546985938056], ComputedExpression[set_index_DataFrame_140546985958816], ComputedExpression[pre_combine_sum_DataFrame_140546985969928]]:140546985909664]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140546985908320]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140546985938056], ComputedExpression[set_index_DataFrame_140546985958816], ComputedExpression[pre_combine_sum_DataFrame_140546985969928]], outputs={ComputedExpression[pre_combine_sum_DataFrame_140546985969928], PlaceholderExpression[placeholder_DataFrame_140546985908320]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140546985959376]]:140546987138912]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140546985969928]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140546985959376]], outputs={ComputedExpression[post_combine_sum_DataFrame_140546985959376]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.temp8e5481dc-912b-4eb1-80a4-a9c800994aee
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7fd3ab86a268> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7fd3ab86a378> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fd3ab86a840> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7fd3ab86a8c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7fd3ab86aa60> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7fd3ab86aae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7fd3ab86abf8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7fd3ab86ac80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7fd3ab86ad08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7fd3ab86ad90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fd3ab880048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7fd3ab86af28> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7fd3ab8800d0> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7fd3a5250518> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmpla3oi5xw.result/[ComputedExpression[post_combine_sum_DataFrame_140546985959376]]:140546987138912/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmpla3oi5xw.result/[ComputedExpression[post_combine_sum_DataFrame_140546985959376]]:140546987138912/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmpla3oi5xw.result/[ComputedExpression[post_combine_sum_DataFrame_140546985959376]]:140546987138912/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpla3oi5xw-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpla3oi5xw-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140546985959376-with-indexes-ParDo-_Unbatch_46))+(WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpla3oi5xw-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpla3oi5xw-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpla3oi5xw-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmpla3oi5xw.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpla3oi5xw-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.temp8e5481dc-912b-4eb1-80a4-a9c800994aee/5562585065983069077_6319af74-796e-4110-8888-b535f44d5d35 to dir: /tmp as tmpla3oi5xw.result-00000-of-00001. Res: FileResult(file_name='/tmp/.temp8e5481dc-912b-4eb1-80a4-a9c800994aee/5562585065983069077_6319af74-796e-4110-8888-b535f44d5d35', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 38 warnings in 257.23 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 6m 31s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/7x2bm6humhidq

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #21

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/21/display/redirect?page=changes>

Changes:

[noreply] Update README.md

[marco.robles] Update README with latest PreCommit Jobs

[marco.robles] Update Postcommit jobs with latest jobs

[marco.robles] Update Performace job tests in readme

[marco.robles] update load job tests with latest updates

[marco.robles] update other jobs test with latest updates

[marco.robles] mismatch links fix

[marco.robles] update trigger phrase for some postCommit jobs

[marco.robles] correct trigger phrases in readme

[marco.robles] add pending jobs to readme

[noreply] Update README.md

[marco.robles] fix broken links in jobs & remove the invalid ones

[noreply] [BEAM-12914] Add missing 3.9 opcodes to type inference. (#16761)

[noreply] [BEAM-13321] Initial BigQueryIO externalization. (#16489)

[noreply] [BEAM-13193] Enable process bundle response elements embedding in Java

[noreply] [BEAM-13830] added a debeziumio_expansion_addr flag to GoSDK (#16780)

[noreply] Apply spotless. (#16783)


------------------------------------------
[...truncated 804.09 KB...]
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 39 warnings in 205.05 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/4273148420993145938_e474dc61-5d58-4152-b93e-1ddbcf68a6e2 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821 as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/4273148420993145938_e474dc61-5d58-4152-b93e-1ddbcf68a6e2', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03364443778991699 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/8486159956238836654_e0b32b1f-6240-479d-86b4-eb8c6415d431 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821 as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/8486159956238836654_e0b32b1f-6240-479d-86b4-eb8c6415d431', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03835606575012207 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/7974773739052588114_d1d7a6c5-26e6-49df-9257-dfbffd9cc6da to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-645eca84-9734-4e04-8251-633733861821 as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp0646b1c1-ea9b-4f1a-bb4c-504e4ec664bc/7974773739052588114_d1d7a6c5-26e6-49df-9257-dfbffd9cc6da', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.031159400939941406 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.0330810546875 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03455066680908203 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.04892778396606445 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.033799171447753906 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140707346406032], ComputedExpression[set_index_DataFrame_140707344273136], ComputedExpression[pre_combine_sum_DataFrame_140707312221656]]:140708500972040]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140707344439224]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140707346406032], ComputedExpression[set_index_DataFrame_140707344273136], ComputedExpression[pre_combine_sum_DataFrame_140707312221656]], outputs={ComputedExpression[pre_combine_sum_DataFrame_140707312221656], PlaceholderExpression[placeholder_DataFrame_140707344439224]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140707344271512]]:140707338389040]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140707312221656]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140707344271512]], outputs={ComputedExpression[post_combine_sum_DataFrame_140707344271512]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.tempfa32b4b8-3089-4660-9721-974c83ac24e0
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7ff8ffc6c268> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7ff8ffc6c378> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7ff8ffc6c840> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7ff8ffc6c8c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7ff8ffc6ca60> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7ff8ffc6cae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7ff8ffc6cbf8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7ff8ffc6cc80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7ff8ffc6cd08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7ff8ffc6cd90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7ff8ffc88048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7ff8ffc6cf28> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7ff8ffc880d0> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7ff8fb8d7f28> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmpikkyw6y3.result/[ComputedExpression[post_combine_sum_DataFrame_140707344271512]]:140707338389040/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmpikkyw6y3.result/[ComputedExpression[post_combine_sum_DataFrame_140707344271512]]:140707338389040/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmpikkyw6y3.result/[ComputedExpression[post_combine_sum_DataFrame_140707344271512]]:140707338389040/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmpikkyw6y3-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpikkyw6y3-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140707344271512-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpikkyw6y3-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpikkyw6y3-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpikkyw6y3-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmpikkyw6y3.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmpikkyw6y3-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.tempfa32b4b8-3089-4660-9721-974c83ac24e0/5562587938112418477_ae356d0d-8517-455a-bf12-4e6d5da06bdf to dir: /tmp as tmpikkyw6y3.result-00000-of-00001. Res: FileResult(file_name='/tmp/.tempfa32b4b8-3089-4660-9721-974c83ac24e0/5562587938112418477_ae356d0d-8517-455a-bf12-4e6d5da06bdf', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5181 deselected, 38 warnings in 209.81 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 5m 41s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/s4e6chj7y7q42

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #20

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/20/display/redirect?page=changes>

Changes:

[mmack] [BEAM-13246] Add support for S3 Bucket Key at the object level (AWS Sdk

[Pablo Estrada] Output successful rows from BQ Streaming Inserts

[schapman] BEAM-13439 Type annotation for ptransform_fn

[noreply] [BEAM-13606] Fail bundles with failed BigTable mutations (#16751)


------------------------------------------
[...truncated 800.06 KB...]
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 39 warnings in 236.54 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     apache_beam.io.filebasedsink:filebasedsink.py:348 Renamed 1 shards in 0.10 seconds.
PASSED                                                                   [ 91%]
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays 
-------------------------------- live log call ---------------------------------
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[move_grouped_columns_to_index_DataFrame_140184668457392]]:140184671521760]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140184667637520]}, partitioning=Arbitrary, ops=[ComputedExpression[move_grouped_columns_to_index_DataFrame_140184668457392]], outputs={ComputedExpression[move_grouped_columns_to_index_DataFrame_140184668457392], PlaceholderExpression[placeholder_DataFrame_140184667637520]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[apply_DataFrame_140184671520192]]:140185556580504]> for Stage[inputs={ComputedExpression[move_grouped_columns_to_index_DataFrame_140184668457392]}, partitioning=Index['airline'], ops=[ComputedExpression[apply_DataFrame_140184671520192]], outputs={ComputedExpression[apply_DataFrame_140184671520192]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2
WARNING  apache_beam.options.pipeline_options:pipeline_options.py:309 Discarding unparseable args: ['--sleep_secs=20', '--kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test']
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7f7f4ddb72f0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7f7f4ddb7400> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f7f4ddb78c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7f7f4ddb7950> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7f7f4ddb7ae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7f7f4ddb7b70> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7f7f4ddb7c80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7f7f4ddb7d08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7f7f4ddb7d90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7f7f4ddb7e18> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f7f4ddb50d0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7f7f4ddb5048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7f7f4ddb5158> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f7f7e03ce10> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-Read-Impulse_10)+(ref_AppliedPTransform_read-table-Read-Map-lambda-at-iobase-py-898-_11))+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_6_split/Write)
INFO     apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:557 Started BigQuery job: <JobReference
                                                                  location: 'US'
                                                                  projectId: 'apache-beam-testing'>
                                                                  bq show -j --format=prettyjson --project_id=apache-beam-testing None
INFO     apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:439 Using location 'US' from table <TableReference
                                                                  datasetId: 'airline_ontime_data'
                                                                  projectId: 'bigquery-samples'
                                                                  tableId: 'flights'> referenced by query 
                                                                   SELECT
                                                                     date,
                                                                     airline,
                                                                     departure_airport,
                                                                     arrival_airport,
                                                                     departure_delay,
                                                                     arrival_delay
                                                                   FROM `bigquery-samples.airline_ontime_data.flights`
                                                                   WHERE date >= '2012-12-23' AND date <= '2012-12-25'
                                                                   
WARNING  apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:886 Dataset apache-beam-testing:beam_temp_dataset_f025373bf8164d378da909fe88198221 does not exist so we will create it as temporary with location=US
INFO     apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:557 Started BigQuery job: <JobReference
                                                                  jobId: 'beam_bq_job_QUERY_BQ_EXPORT_JOB_2fe9caa0-c_1644343832_782'
                                                                  location: 'US'
                                                                  projectId: 'apache-beam-testing'>
                                                                  bq show -j --format=prettyjson --project_id=apache-beam-testing beam_bq_job_QUERY_BQ_EXPORT_JOB_2fe9caa0-c_1644343832_782
INFO     root:bigquery_tools.py:625 Job status: RUNNING
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.bigquery_tools:bigquery_tools.py:557 Started BigQuery job: <JobReference
                                                                  jobId: 'beam_bq_job_EXPORT_BQ_EXPORT_JOB_2fe9caa0-c_1644343838_536'
                                                                  location: 'US'
                                                                  projectId: 'apache-beam-testing'>
                                                                  bq show -j --format=prettyjson --project_id=apache-beam-testing beam_bq_job_EXPORT_BQ_EXPORT_JOB_2fe9caa0-c_1644343838_536
INFO     root:bigquery_tools.py:625 Job status: RUNNING
INFO     root:bigquery_tools.py:625 Job status: DONE
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03880000114440918 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((((((((((((((((ref_PCollection_PCollection_6_split/Read)+(read table/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-PassThrough-ParDo-PassThrough-_16))+(ref_PCollection_PCollection_9/Write))+(ref_AppliedPTransform_assign-timestamp_23))+(ref_AppliedPTransform_set-schema-Map-lambda-at-core-py-2836-_25))+(ref_AppliedPTransform_daily-windows_26))+(ref_AppliedPTransform_BatchElements-daily-BatchElements-ParDo-_WindowAwareBatchingDoFn-_29))+(ref_AppliedPTransform_BatchElements-daily-Map-lambda-at-schemas-py-140-_30))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__33))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__35))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__38))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__39))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__40))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__43))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/[ComputedExpression[apply_DataFrame_140184671520192]]:140185556580504/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__44))+(ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/[ComputedExpression[apply_DataFrame_140184671520192]]:140185556580504/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03973245620727539 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/[ComputedExpression[apply_DataFrame_140184671520192]]:140185556580504/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/4273148420993145938_3f75d6b1-3a6a-496e-826d-066a631c5db4 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/4273148420993145938_3f75d6b1-3a6a-496e-826d-066a631c5db4', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.04448843002319336 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/8486159956238836654_9e138ac0-96ec-477d-b772-db95ef70d26f to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/8486159956238836654_9e138ac0-96ec-477d-b772-db95ef70d26f', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.03538680076599121 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/7974773739052588114_0d264aaf-ab8f-4a29-a6df-4968048362b6 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-9c45ba2f-2ec0-4322-983a-6106e1837e6c as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.tempe58639c7-1045-4fdc-9f74-1b69234c89d2/7974773739052588114_0d264aaf-ab8f-4a29-a6df-4968048362b6', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.033559560775756836 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.035944223403930664 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.03842592239379883 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.038726806640625 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.031705617904663086 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140184300578576], ComputedExpression[set_index_DataFrame_140184670703064], ComputedExpression[pre_combine_sum_DataFrame_140185558535696]]:140184667797824]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140184672062488]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140184300578576], ComputedExpression[set_index_DataFrame_140184670703064], ComputedExpression[pre_combine_sum_DataFrame_140185558535696]], outputs={PlaceholderExpression[placeholder_DataFrame_140184672062488], ComputedExpression[pre_combine_sum_DataFrame_140185558535696]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140184668836304]]:140184667797152]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140185558535696]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140184668836304]], outputs={ComputedExpression[post_combine_sum_DataFrame_140184668836304]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.tempb157fa07-d54e-40f6-848c-8c677d3e1111
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7f7f4ddb72f0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7f7f4ddb7400> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7f7f4ddb78c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7f7f4ddb7950> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7f7f4ddb7ae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7f7f4ddb7b70> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7f7f4ddb7c80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7f7f4ddb7d08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7f7f4ddb7d90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7f7f4ddb7e18> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7f7f4ddb50d0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7f7f4ddb5048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7f7f4ddb5158> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7f7f4950f4a8> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmp3sgsjuww.result/[ComputedExpression[post_combine_sum_DataFrame_140184668836304]]:140184667797152/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmp3sgsjuww.result/[ComputedExpression[post_combine_sum_DataFrame_140184668836304]]:140184667797152/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmp3sgsjuww.result/[ComputedExpression[post_combine_sum_DataFrame_140184668836304]]:140184667797152/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp3sgsjuww-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp3sgsjuww-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140184668836304-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp3sgsjuww-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp3sgsjuww-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp3sgsjuww-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmp3sgsjuww.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp3sgsjuww-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.tempb157fa07-d54e-40f6-848c-8c677d3e1111/5562587401826285902_44ffc710-ab00-40cf-bcdb-6a084390d5fd to dir: /tmp as tmp3sgsjuww.result-00000-of-00001. Res: FileResult(file_name='/tmp/.tempb157fa07-d54e-40f6-848c-8c677d3e1111/5562587401826285902_44ffc710-ab00-40cf-bcdb-6a084390d5fd', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 38 warnings in 235.97 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 5m 14s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/42x6nqjc7m4lw

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org


Build failed in Jenkins: beam_PostCommit_Python_Examples_Direct #19

Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/19/display/redirect?page=changes>

Changes:

[Ismaël Mejía] [BEAM-13839] Upgrade zstd-jni to version 1.5.2-1

[mmack] [BEAM-13840] Fix usage of legacy rawtypes in AWS modules


------------------------------------------
[...truncated 804.34 KB...]
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/examples/dataframe/flight_delays.py>:45: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.
    return airline_df[at_top_airports].mean()

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py37.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 39 warnings in 210.60 seconds =====

> Task :sdks:python:test-suites:direct:py36:examples
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Impulse_4)+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-FlatMap-lambda-at-core-py-3228-_5))+(ref_AppliedPTransform_read-table-FilesToRemoveImpulse-Map-decode-_7))+(ref_AppliedPTransform_read-table-MapFilesToRemove_8))+(ref_PCollection_PCollection_4/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (((ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Impulse_18)+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-FlatMap-lambda-at-core-py-3228-_19))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-Create-Map-decode-_21))+(ref_AppliedPTransform_read-table-_PassThroughThenCleanup-ParDo-RemoveExtractedFiles-_22)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.05219006538391113 seconds.
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((ToPCollection(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/[ComputedExpression[apply_DataFrame_140404000645416]]:140404001314408/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__46))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__47))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__48))+(ref_AppliedPTransform_ToPCollection-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__50))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__54))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/Flatten/Write/0))+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__55))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__57))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__59))+(WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e/output.csv/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---gs-temp-storage-for-end-to-end-tests-temp-it-flight_delays__61)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/4273148420993145938_5cdea6c1-2ceb-424d-a09c-1c812620d3ac to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e as output.csv-2012-12-24T00:00:00-2012-12-25T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/4273148420993145938_5cdea6c1-2ceb-424d-a09c-1c812620d3ac', shard_index=-1, total_shards=0, window=[1356307200.0, 1356393600.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356307200.0, 1356393600.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.06291961669921875 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/7974773739052588114_b74aa4b6-2cf2-469e-99ed-0679c0508b16 to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e as output.csv-2012-12-23T00:00:00-2012-12-24T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/7974773739052588114_b74aa4b6-2cf2-469e-99ed-0679c0508b16', shard_index=-1, total_shards=0, window=[1356220800.0, 1356307200.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356220800.0, 1356307200.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.08235979080200195 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/8486159956238836654_1bee10b8-f3c3-4f4a-bd4e-1caf7f209d7a to dir: gs://temp-storage-for-end-to-end-tests/temp-it/flight_delays_it-ef6ca8c6-d1b7-428c-aa70-ced0a7b0f77e as output.csv-2012-12-25T00:00:00-2012-12-26T00:00:00-00000-of-00001. Res: FileResult(file_name='gs://temp-storage-for-end-to-end-tests/temp-it/.temp6259d7bc-78fc-4245-9ff6-77db030d45b4/8486159956238836654_1bee10b8-f3c3-4f4a-bd4e-1caf7f209d7a', shard_index=-1, total_shards=0, window=[1356393600.0, 1356480000.0), pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window [1356393600.0, 1356480000.0)
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 0 files in 0.04333305358886719 seconds.
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.07561683654785156 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.06525492668151855 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 1 files in 0.04582476615905762 seconds.
INFO     apache_beam.io.gcp.gcsio:gcsio.py:559 Starting the size estimation of the input
INFO     apache_beam.io.gcp.gcsio:gcsio.py:575 Finished listing 3 files in 0.07879447937011719 seconds.
PASSED                                                                   [ 95%]
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics 
-------------------------------- live log call ---------------------------------
INFO     root:pipeline.py:188 Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[set_column_DataFrame_140403998550168], ComputedExpression[set_index_DataFrame_140403999774368], ComputedExpression[pre_combine_sum_DataFrame_140403999774424]]:140403998550336]> for Stage[inputs={PlaceholderExpression[placeholder_DataFrame_140402843130568]}, partitioning=Arbitrary, ops=[ComputedExpression[set_column_DataFrame_140403998550168], ComputedExpression[set_index_DataFrame_140403999774368], ComputedExpression[pre_combine_sum_DataFrame_140403999774424]], outputs={ComputedExpression[pre_combine_sum_DataFrame_140403999774424], PlaceholderExpression[placeholder_DataFrame_140402843130568]}]
INFO     root:transforms.py:182 Computing dataframe stage <ComputeStage(PTransform) label=[[ComputedExpression[post_combine_sum_DataFrame_140402843418408]]:140404004701296]> for Stage[inputs={ComputedExpression[pre_combine_sum_DataFrame_140403999774424]}, partitioning=Index, ops=[ComputedExpression[post_combine_sum_DataFrame_140402843418408]], outputs={ComputedExpression[post_combine_sum_DataFrame_140402843418408]}]
INFO     apache_beam.io.fileio:fileio.py:555 Added temporary directory /tmp/.temp4b8045c2-5dab-40b4-8409-ea6763ba9747
WARNING  root:environments.py:374 Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
INFO     root:environments.py:380 Default Python SDK image for environment is apache/beam_python3.6_sdk:2.37.0.dev
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function annotate_downstream_side_inputs at 0x7fb21a4d82f0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function fix_side_input_pcoll_coders at 0x7fb21a4d8400> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function pack_combiners at 0x7fb21a4d88c8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function lift_combiners at 0x7fb21a4d8950> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_sdf at 0x7fb21a4d8ae8> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function expand_gbk at 0x7fb21a4d8b70> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sink_flattens at 0x7fb21a4d8c80> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function greedily_fuse at 0x7fb21a4d8d08> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function read_to_impulse at 0x7fb21a4d8d90> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function impulse_to_input at 0x7fb21a4d8e18> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function sort_stages at 0x7fb21a4f40d0> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function setup_timer_mapping at 0x7fb21a4f4048> ====================
INFO     apache_beam.runners.portability.fn_api_runner.translations:translations.py:678 ==================== <function populate_data_channel_coders at 0x7fb21a4f4158> ====================
INFO     apache_beam.runners.worker.statecache:statecache.py:172 Creating state cache with size 100
INFO     apache_beam.runners.portability.fn_api_runner.worker_handlers:worker_handlers.py:894 Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7fb21633bac8> for environment ref_Environment_default_environment_1 (beam:env:embedded_python:v1, b'')
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((ref_AppliedPTransform_Read-Read-Impulse_4)+(ref_AppliedPTransform_Read-Read-Map-lambda-at-iobase-py-898-_5))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_2_split/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((((ref_PCollection_PCollection_2_split/Read)+(Read/Read/SDFBoundedSourceReader/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_Split_8))+(ref_AppliedPTransform_ToRows_9))+(ref_AppliedPTransform_BatchElements-words-BatchElements-ParDo-_GlobalWindowsBatchingDoFn-_12))+(ref_AppliedPTransform_BatchElements-words-Map-lambda-at-schemas-py-140-_13))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-set_column_DataFr_16))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-set_column_DataFr_18))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__21))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__22))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__23))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__26))+(ToPCollection(df) - /tmp/tmp__q90bah.result/[ComputedExpression[post_combine_sum_DataFrame_140402843418408]]:140404004701296/CoGroupByKey/CoGroupByKeyImpl/Flatten/Transcode/0))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__27))+(ToPCollection(df) - /tmp/tmp__q90bah.result/[ComputedExpression[post_combine_sum_DataFrame_140402843418408]]:140404004701296/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((((((((((((ToPCollection(df) - /tmp/tmp__q90bah.result/[ComputedExpression[post_combine_sum_DataFrame_140402843418408]]:140404004701296/CoGroupByKey/CoGroupByKeyImpl/GroupByKey/Read)+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__29))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__30))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__31))+(ref_AppliedPTransform_ToPCollection-df---tmp-tmp__q90bah-result-ComputedExpression-post_combine_sum__33))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp__q90bah-result-WriteToFiles-ParDo-_WriteUnshardedRe_37))+(ref_AppliedPTransform_Unbatch-post_combine_sum_DataFrame_140402843418408-with-indexes-ParDo-_Unbatch_46))+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp__q90bah-result-WriteToFiles-ParDo-_AppendShardedDes_38))+(WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/Flatten/Write/0))+(WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/GroupRecordsByDestinationAndShard/Write))+(ref_AppliedPTransform_Filter-lambda-at-wordcount-py-80-_47))+(ref_AppliedPTransform_Map-lambda-at-wordcount-py-81-_48))+(ref_AppliedPTransform_Map-print-_49)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/GroupRecordsByDestinationAndShard/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp__q90bah-result-WriteToFiles-ParDo-_WriteShardedReco_40))+(WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/Flatten/Write/1)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running ((WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/Flatten/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp__q90bah-result-WriteToFiles-Map-lambda-at-fileio-py_42))+(WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/GroupTempFilesByDestination/Write)
INFO     apache_beam.runners.portability.fn_api_runner.fn_runner:fn_runner.py:621 Running (WriteToPandas(df) - /tmp/tmp__q90bah.result/WriteToFiles/GroupTempFilesByDestination/Read)+(ref_AppliedPTransform_WriteToPandas-df---tmp-tmp__q90bah-result-WriteToFiles-ParDo-_MoveTempFilesInt_44)
INFO     apache_beam.io.fileio:fileio.py:642 Moving temporary file /tmp/.temp4b8045c2-5dab-40b4-8409-ea6763ba9747/5562584426033259977_49bb9c8b-f177-46f6-870d-b83964585b66 to dir: /tmp as tmp__q90bah.result-00000-of-00001. Res: FileResult(file_name='/tmp/.temp4b8045c2-5dab-40b4-8409-ea6763ba9747/5562584426033259977_49bb9c8b-f177-46f6-870d-b83964585b66', shard_index=-1, total_shards=0, window=GlobalWindow, pane=None, destination=None)
INFO     apache_beam.io.fileio:fileio.py:665 Checking orphaned temporary files for destination None and window GlobalWindow
INFO     apache_beam.io.fileio:fileio.py:678 Some files may be left orphaned in the temporary folder: []
PASSED                                                                   [100%]

=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c
    self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf'))  # pylint: disable=anomalous-backslash-in-string

apache_beam/io/filesystems_test.py:62
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d
    self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'),  # pylint: disable=anomalous-backslash-in-string

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:63: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    dataset_ref = client.dataset(unique_dataset_name, project=project)

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2138: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2144: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    experiments = p.options.view_as(DebugOptions).experiments or []

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1128: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = p.options.view_as(GoogleCloudOptions).temp_location

apache_beam/examples/complete/game/hourly_team_score_it_test.py::HourlyTeamScoreIT::test_hourly_team_score_output_checksum_on_small_input
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py>:1130: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).job_name or 'AUTOMATIC_JOB_NAME')

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2437: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    temp_location = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2439: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    job_name = pcoll.pipeline.options.view_as(GoogleCloudOptions).job_name

apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_side_input_it_test.py::BigQuerySideInputIT::test_bigquery_side_input_it
apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2470: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    | _PassThroughThenCleanup(files_to_remove_pcoll))

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/bigquery.py>:2134: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    self.table_reference.projectId = pcoll.pipeline.options.view_as(

apache_beam/examples/cookbook/bigquery_tornadoes_it_test.py::BigqueryTornadoesIT::test_bigquery_tornadoes_it
apache_beam/examples/cookbook/filters_test.py::FiltersTest::test_filters_output_bigquery_matcher
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/gcp/tests/utils.py>:100: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = client.dataset(dataset_id).table(table_id)

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/dataframe/io.py>:632: FutureWarning: WriteToFiles is experimental.
    sink=lambda _: _WriteToPandasFileSink(

apache_beam/examples/dataframe/flight_delays_it_test.py::FlightDelaysTest::test_flight_delays
apache_beam/examples/dataframe/wordcount_test.py::WordCountTest::test_basics
  <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/apache_beam/io/fileio.py>:550: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
    p.options.view_as(GoogleCloudOptions).temp_location or

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/pytest_postCommitExamples-direct-py36.xml> -
===== 22 passed, 1 skipped, 5178 deselected, 38 warnings in 219.10 seconds =====

FAILURE: Build failed with an exception.

* Where:
Script '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/ws/src/sdks/python/test-suites/direct/common.gradle'> line: 92

* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py38:examples'.
> Process 'command 'sh'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 6m 29s
21 actionable tasks: 15 executed, 4 from cache, 2 up-to-date

Publishing build scan...
https://gradle.com/s/evzfcfe2ozfxc

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure


---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org