You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/03/06 12:09:10 UTC
Build failed in Jenkins: beam_PostCommit_Python37 #1824
See <https://builds.apache.org/job/beam_PostCommit_Python37/1824/display/redirect>
Changes:
------------------------------------------
[...truncated 1.69 MB...]
DEBUG:urllib3.connectionpool:http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/844138762903-compute@developer.gserviceaccount.com/token HTTP/1.1" 200 192
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): bigquery.googleapis.com:443
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/queries/05910e06-56e6-419c-ad13-bf7c63cd9b00?maxResults=0&location=US HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon9e8b5990_63c8_4bdc_8c22_ccda9345defe/data HTTP/1.1" 200 None
INFO:apache_beam.io.gcp.tests.bigquery_matcher:Read from given query (SELECT bytes, date, time FROM `python_query_to_table_15834965235215.output_table`;), total rows 4
INFO:apache_beam.io.gcp.tests.bigquery_matcher:Generate checksum: 24de460c4d344a4b77ccc4cc1acb7b7ffc11a214
DEBUG:root:gcs_location is empty, using temp_location instead
DEBUG:root:Unhandled type_constraint: Const[RemoveJsonFiles]
DEBUG:root:Unhandled type_constraint: Union[]
DEBUG:root:Unhandled type_constraint: Union[]
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function annotate_downstream_side_inputs at 0x7f4e6192cea0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:15 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey_24\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function fix_side_input_pcoll_coders at 0x7f4e61929048> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:15 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey_24\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function lift_combiners at 0x7f4e619290d0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:15 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey_24\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_sdf at 0x7f4e61929158> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:17 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\n must follow: read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey_24\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function expand_gbk at 0x7f4e619291e0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:18 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\n must follow: read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: \n downstream_side_inputs: ', 'write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\n must follow: write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write\n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sink_flattens at 0x7f4e619292f0> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:18 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\n must follow: read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction\n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9\n read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14\n read/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15\n read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18\n write/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19\n write/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21\n write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: \n downstream_side_inputs: ', 'write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\n must follow: write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write\n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29\n write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ', 'ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31\n write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function greedily_fuse at 0x7f4e61929378> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [9, 4, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\nread/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function read_to_impulse at 0x7f4e61929400> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [9, 4, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/Impulse:beam:transform:impulse:v1\nread/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/Impulse:beam:transform:impulse:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function impulse_to_input at 0x7f4e61929488> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [9, 4, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Impulse:beam:source:runner:0.1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/Impulse:beam:source:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function inject_timer_pcollections at 0x7f4e61929620> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [9, 4, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Impulse:beam:source:runner:0.1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/Impulse:beam:source:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function sort_stages at 0x7f4e619296a8> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [4, 9, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/Impulse:beam:source:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', '((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Impulse:beam:source:runner:0.1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ']
INFO:apache_beam.runners.portability.fn_api_runner_transforms:==================== <function window_pcollection_coders at 0x7f4e61929730> ====================
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:4 [4, 9, 4, 4]
DEBUG:apache_beam.runners.portability.fn_api_runner_transforms:Stages: ['(((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction:beam:transform:sdf_pair_with_restriction:v1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction:beam:transform:sdf_split_and_size_restrictions:v1\nref_PCollection_PCollection_1_split/Write:beam:sink:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/Impulse:beam:source:runner:0.1\n must follow: \n downstream_side_inputs: ref_PCollection_PCollection_4', '((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n ref_PCollection_PCollection_1_split/Read:beam:source:runner:0.1\nread/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process:beam:transform:sdf_process_sized_element_and_restrictions:v1\nread/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/AppendDestination:beam:transform:pardo:v1\nref_PCollection_PCollection_4/Write:beam:sink:runner:0.1\nwrite/_StreamToBigQuery/AddInsertIds:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/AddRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write:beam:sink:runner:0.1\n must follow: (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)\n downstream_side_inputs: ref_PCollection_PCollection_4', '(((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)\n read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Map(decode):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/ParDo(RemoveJsonFiles):beam:transform:pardo:v1\nread/_PassThroughThenCleanup/Create/Impulse:beam:source:runner:0.1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ', '(((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)\n write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read:beam:source:runner:0.1\nwrite/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\nwrite/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys:beam:transform:pardo:v1\nwrite/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn):beam:transform:pardo:v1\n must follow: ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)\n downstream_side_inputs: ']
INFO:apache_beam.runners.worker.statecache:Creating state cache with size 100
INFO:apache_beam.runners.portability.fn_api_runner:Created Worker handler <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f4e61409da0> for environment urn: "beam:env:embedded_python:v1"
ERROR:apache_beam.runners.portability.fn_api_runner:created 1 workers {'worker_2': <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f4e61409da0>}
INFO:apache_beam.runners.portability.fn_api_runner:Running (((ref_AppliedPTransform_read/Read/_SDFBoundedSourceWrapper/Impulse_5)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction))+(ref_PCollection_PCollection_1_split/Write)
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataOutputOperation ref_PCollection_PCollection_1_split/Write >
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction output_tags=['out'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction.out0, coder=WindowedValueCoder[TupleCoder[TupleCoder[BytesCoder, TupleCoder[LengthPrefixCoder[DillCoder], LengthPrefixCoder[FastPrimitivesCoder]]], FloatCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction output_tags=['out'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction.out0, coder=WindowedValueCoder[TupleCoder[BytesCoder, TupleCoder[DillCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataInputOperation read/Read/_SDFBoundedSourceWrapper/Impulse receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/Impulse.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.io.gcp.bigquery_tools:Query SELECT * FROM (SELECT "apple" as fruit) UNION ALL (SELECT "orange" as fruit) does not reference any tables.
WARNING:apache_beam.io.gcp.bigquery_tools:Dataset apache-beam-testing:temp_dataset_e4e4c68e78854dbf8ef9921593061c42 does not exist so we will create it as temporary with location=None
INFO:root:Job status: RUNNING
INFO:root:Job status: DONE
INFO:root:Job status: RUNNING
INFO:root:Job status: DONE
DEBUG:apache_beam.io.filesystem:Listing files in 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-'
DEBUG:apache_beam.io.filesystem:translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-*.json' -> 'gs://temp\\-storage\\-for\\-end\\-to\\-end\\-tests/temp\\-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery\\-table\\-dump\\-[^/\\\\]*\\.json'
INFO:apache_beam.io.gcp.gcsio:Starting the size estimation of the input
INFO:apache_beam.io.gcp.gcsio:Finished listing 1 files in 0.04346585273742676 seconds.
DEBUG:apache_beam.io.filesystem:translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-000000000000.json' -> 'gs://temp\\-storage\\-for\\-end\\-to\\-end\\-tests/temp\\-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery\\-table\\-dump\\-000000000000\\.json'
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataInputOperation read/Read/_SDFBoundedSourceWrapper/Impulse receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/Impulse.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction output_tags=['out'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction.out0, coder=WindowedValueCoder[TupleCoder[BytesCoder, TupleCoder[DillCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction output_tags=['out'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction.out0, coder=WindowedValueCoder[TupleCoder[TupleCoder[BytesCoder, TupleCoder[LengthPrefixCoder[DillCoder], LengthPrefixCoder[FastPrimitivesCoder]]], FloatCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataOutputOperation ref_PCollection_PCollection_1_split/Write >
DEBUG:apache_beam.runners.portability.fn_api_runner:Wait for the bundle bundle_9 to finish.
ERROR:apache_beam.runners.portability.fn_api_runner:created 1 workers {'worker_2': <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f4e61409da0>}
INFO:apache_beam.runners.portability.fn_api_runner:Running ((((((((ref_PCollection_PCollection_1_split/Read)+(read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9))+(ref_AppliedPTransform_write/_StreamToBigQuery/AppendDestination_18))+(ref_PCollection_PCollection_4/Write))+(ref_AppliedPTransform_write/_StreamToBigQuery/AddInsertIds_19))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_21))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_23))+(write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write)
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataOutputOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write >
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataOutputOperation ref_PCollection_PCollection_4/Write >
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps).out0, coder=WindowedValueCoder[TupleCoder[LengthPrefixCoder[FastPrimitivesCoder], LengthPrefixCoder[FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys.out0, coder=WindowedValueCoder[TupleCoder[VarIntCoder, TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/AddInsertIds output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/AddInsertIds.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/AppendDestination output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/AppendDestination.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough) output_tags=['None', 'cleanup_signal'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1], SingletonConsumerSet[read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough).out1, coder=WindowedValueCoder[LengthPrefixCoder[FastPrimitivesCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <SdfProcessSizedElements read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process output_tags=['None'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process.out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataInputOperation ref_PCollection_PCollection_1_split/Read receivers=[SingletonConsumerSet[ref_PCollection_PCollection_1_split/Read.out0, coder=WindowedValueCoder[TupleCoder[TupleCoder[BytesCoder, TupleCoder[LengthPrefixCoder[DillCoder], LengthPrefixCoder[FastPrimitivesCoder]]], FloatCoder]], len(consumers)=1]]>
DEBUG:apache_beam.io.filesystem:translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-000000000000.json' -> 'gs://temp\\-storage\\-for\\-end\\-to\\-end\\-tests/temp\\-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery\\-table\\-dump\\-000000000000\\.json'
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataInputOperation ref_PCollection_PCollection_1_split/Read receivers=[SingletonConsumerSet[ref_PCollection_PCollection_1_split/Read.out0, coder=WindowedValueCoder[TupleCoder[TupleCoder[BytesCoder, TupleCoder[LengthPrefixCoder[DillCoder], LengthPrefixCoder[FastPrimitivesCoder]]], FloatCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <SdfProcessSizedElements read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process output_tags=['None'], receivers=[SingletonConsumerSet[read/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process.out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough) output_tags=['None', 'cleanup_signal'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1], SingletonConsumerSet[read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough).out1, coder=WindowedValueCoder[LengthPrefixCoder[FastPrimitivesCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/AppendDestination output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/AppendDestination.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/AddInsertIds output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/AddInsertIds.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/AddRandomKeys.out0, coder=WindowedValueCoder[TupleCoder[VarIntCoder, TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps).out0, coder=WindowedValueCoder[TupleCoder[LengthPrefixCoder[FastPrimitivesCoder], LengthPrefixCoder[FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataOutputOperation ref_PCollection_PCollection_4/Write >
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataOutputOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write >
DEBUG:apache_beam.runners.portability.fn_api_runner:Wait for the bundle bundle_10 to finish.
ERROR:apache_beam.runners.portability.fn_api_runner:created 1 workers {'worker_2': <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f4e61409da0>}
INFO:apache_beam.runners.portability.fn_api_runner:Running (((ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Impulse_11)+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>)_12))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/Create/Map(decode)_14))+(ref_AppliedPTransform_read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15)
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles) output_tags=['None'], receivers=[ConsumerSet[read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/_PassThroughThenCleanup/Create/Map(decode) output_tags=['None'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/Map(decode).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>) output_tags=['None'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>).out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataInputOperation read/_PassThroughThenCleanup/Create/Impulse receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/Impulse.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.io.filesystem:Listing files in 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-'
DEBUG:apache_beam.io.filesystem:translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-*.json' -> 'gs://temp\\-storage\\-for\\-end\\-to\\-end\\-tests/temp\\-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery\\-table\\-dump\\-[^/\\\\]*\\.json'
INFO:apache_beam.io.gcp.gcsio:Starting the size estimation of the input
INFO:apache_beam.io.gcp.gcsio:Finished listing 1 files in 0.03183460235595703 seconds.
DEBUG:root:RemoveJsonFiles: matched 1 files
DEBUG:apache_beam.io.filesystem:translate_pattern: 'gs://temp-storage-for-end-to-end-tests/temp-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery-table-dump-000000000000.json' -> 'gs://temp\\-storage\\-for\\-end\\-to\\-end\\-tests/temp\\-it/6d78a97f9e9c42d0b19ef3e231d2beb2/bigquery\\-table\\-dump\\-000000000000\\.json'
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataInputOperation read/_PassThroughThenCleanup/Create/Impulse receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/Impulse.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>) output_tags=['None'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2643>).out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/_PassThroughThenCleanup/Create/Map(decode) output_tags=['None'], receivers=[SingletonConsumerSet[read/_PassThroughThenCleanup/Create/Map(decode).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles) output_tags=['None'], receivers=[ConsumerSet[read/_PassThroughThenCleanup/ParDo(RemoveJsonFiles).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0]]>
DEBUG:apache_beam.runners.portability.fn_api_runner:Wait for the bundle bundle_11 to finish.
ERROR:apache_beam.runners.portability.fn_api_runner:created 1 workers {'worker_2': <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f4e61409da0>}
INFO:apache_beam.runners.portability.fn_api_runner:Running (((write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_28))+(ref_AppliedPTransform_write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_29))+(ref_AppliedPTransform_write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_31)
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) output_tags=['FailedRows', 'None'], receivers=[ConsumerSet[write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0], ConsumerSet[write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn).out1, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DoOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps).out0, coder=WindowedValueCoder[TupleCoder[VarIntCoder, TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:start <DataInputOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read.out0, coder=WindowedValueCoder[TupleCoder[LengthPrefixCoder[FastPrimitivesCoder], IterableCoder[LengthPrefixCoder[FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.io.gcp.bigquery:Creating or getting table <TableReference
datasetId: 'python_query_to_table_15834965301170'
projectId: 'apache-beam-testing'
tableId: 'output_table'> with schema {'fields': [{'name': 'fruit', 'type': 'STRING', 'mode': 'NULLABLE'}]}.
DEBUG:apache_beam.io.gcp.bigquery_tools:Created the table with id output_table
INFO:apache_beam.io.gcp.bigquery_tools:Created table apache-beam-testing.python_query_to_table_15834965301170.output_table with schema <TableSchema
fields: [<TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'fruit'
type: 'STRING'>]>. Result: <Table
creationTime: 1583496544981
etag: 'JyIUGuKxX8ZRZuN158SbFg=='
id: 'apache-beam-testing:python_query_to_table_15834965301170.output_table'
kind: 'bigquery#table'
lastModifiedTime: 1583496545038
location: 'US'
numBytes: 0
numLongTermBytes: 0
numRows: 0
schema: <TableSchema
fields: [<TableFieldSchema
fields: []
mode: 'NULLABLE'
name: 'fruit'
type: 'STRING'>]>
selfLink: 'https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/python_query_to_table_15834965301170/tables/output_table'
tableReference: <TableReference
datasetId: 'python_query_to_table_15834965301170'
projectId: 'apache-beam-testing'
tableId: 'output_table'>
type: 'TABLE'>.
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DataInputOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read.out0, coder=WindowedValueCoder[TupleCoder[LengthPrefixCoder[FastPrimitivesCoder], IterableCoder[LengthPrefixCoder[FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps).out0, coder=WindowedValueCoder[TupleCoder[VarIntCoder, TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys output_tags=['None'], receivers=[SingletonConsumerSet[write/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]]], len(consumers)=1]]>
DEBUG:apache_beam.runners.worker.bundle_processor:finish <DoOperation write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) output_tags=['FailedRows', 'None'], receivers=[ConsumerSet[write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn).out0, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0], ConsumerSet[write/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn).out1, coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=0]]>
DEBUG:apache_beam.io.gcp.bigquery:Attempting to flush to all destinations. Total buffered: 2
DEBUG:apache_beam.io.gcp.bigquery:Flushing data to apache-beam-testing:python_query_to_table_15834965301170.output_table. Total 2 rows.
DEBUG:apache_beam.runners.portability.fn_api_runner:Wait for the bundle bundle_12 to finish.
INFO:apache_beam.io.gcp.tests.bigquery_matcher:Attempting to perform query SELECT fruit from `python_query_to_table_15834965301170.output_table`; to BQ
DEBUG:google.auth.transport._http_client:Making request: GET http://169.254.169.254
DEBUG:google.auth.transport._http_client:Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id
DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
DEBUG:google.auth.transport.requests:Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): metadata.google.internal:80
DEBUG:urllib3.connectionpool:http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144
DEBUG:google.auth.transport.requests:Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/844138762903-compute@developer.gserviceaccount.com/token
DEBUG:urllib3.connectionpool:http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/844138762903-compute@developer.gserviceaccount.com/token HTTP/1.1" 200 192
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): bigquery.googleapis.com:443
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/queries/d432c999-87d9-4f44-97a5-51d2dafd16ab?maxResults=0&location=US HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://bigquery.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anond563d36c_f51a_49fe_9f8d_444901f67969/data HTTP/1.1" 200 None
INFO:apache_beam.io.gcp.tests.bigquery_matcher:Read from given query (SELECT fruit from `python_query_to_table_15834965301170.output_table`;), total rows 2
INFO:apache_beam.io.gcp.tests.bigquery_matcher:Generate checksum: 158a8ea1c254fcf40d4ed3e7c0242c3ea0a29e72
test_datastore_write_limit (apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT) ... ok
test_streaming_data_only (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_streaming_with_attributes (apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest) ... ok
test_bigquery_read_1M_python (apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT) ... ok
test_bigquery_read_custom_1M_python (apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT) ... ok
test_iobase_source (apache_beam.io.gcp.bigquery_read_it_test.ReadTests) ... ok
test_native_source (apache_beam.io.gcp.bigquery_read_it_test.ReadTests) ... ok
test_iobase_source (apache_beam.io.gcp.bigquery_read_it_test.ReadNewTypesTests) ... ok
test_native_source (apache_beam.io.gcp.bigquery_read_it_test.ReadNewTypesTests) ... ok
test_big_query_write (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_new_types (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_schema_autodetect (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_big_query_write_without_schema (apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests) ... ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_big_query_legacy_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_new_types (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_new_types_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... ok
test_big_query_standard_sql_kms_key_native (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... SKIP: This test doesn't work on DirectRunner.
----------------------------------------------------------------------
XML: nosetests-postCommitIT-direct-py37.xml
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 19 tests in 56.257s
OK (SKIP=1)
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/py37/build.gradle'> line: 60
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 28s
82 actionable tasks: 60 executed, 22 from cache
Publishing build scan...
https://gradle.com/s/nljgw4ziyy4gs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Python37 #1827
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python37/1827/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #1826
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python37/1826/display/redirect?page=changes>
Changes:
[suztomo] Google-cloud-bigquery 1.108.0
------------------------------------------
[...truncated 142.65 KB...]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:190)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:58)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:176)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.visitArchiveFile(ShadowCopyAction.groovy:286)
at sun.reflect.GeneratedMethodAccessor476.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:104)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:326)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:352)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:68)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:176)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction$_processArchive_closure3.doCall(ShadowCopyAction.groovy:265)
at sun.reflect.GeneratedMethodAccessor466.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:104)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:326)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1041)
at groovy.lang.Closure.call(Closure.java:411)
at groovy.lang.Closure.call(Closure.java:427)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2296)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2281)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2322)
at org.codehaus.groovy.runtime.dgm$186.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:246)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:55)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.processArchive(ShadowCopyAction.groovy:263)
at sun.reflect.GeneratedMethodAccessor520.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:190)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:58)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:168)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.visitFile(ShadowCopyAction.groovy:248)
at sun.reflect.GeneratedMethodAccessor420.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:190)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:58)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:168)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$BaseStreamAction.processFile(ShadowCopyAction.groovy:183)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1$1.processFile(NormalizingCopyActionDecorator.java:66)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1$1.processFile(DuplicateHandlingCopyActionDecorator.java:60)
at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.processFile(CopyFileVisitorImpl.java:62)
at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.visitFile(CopyFileVisitorImpl.java:46)
at org.gradle.api.internal.file.collections.AbstractSingletonFileTree.visit(AbstractSingletonFileTree.java:36)
at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:118)
at org.gradle.api.internal.file.CompositeFileTree.visit(CompositeFileTree.java:93)
at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:39)
at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:24)
at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:693)
at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:695)
at org.gradle.api.internal.file.copy.DefaultCopySpec.walk(DefaultCopySpec.java:499)
at org.gradle.api.internal.file.copy.CopySpecBackedCopyActionProcessingStream.process(CopySpecBackedCopyActionProcessingStream.java:38)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1.process(DuplicateHandlingCopyActionDecorator.java:44)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1.process(NormalizingCopyActionDecorator.java:57)
at org.gradle.api.internal.file.copy.CopyActionProcessingStream$process.call(Unknown Source)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$2.execute(ShadowCopyAction.groovy:110)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$2$execute.call(Unknown Source)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction.withResource(ShadowCopyAction.groovy:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:104)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:216)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction.execute(ShadowCopyAction.groovy:107)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:53)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:42)
at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:40)
at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:179)
at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar.copy(ShadowJar.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:103)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:48)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:41)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:705)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:672)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:338)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:327)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:312)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$200(ExecuteActionsTaskExecuter.java:75)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:158)
at org.gradle.internal.execution.impl.steps.ExecuteStep.execute(ExecuteStep.java:46)
at org.gradle.internal.execution.impl.steps.CancelExecutionStep.execute(CancelExecutionStep.java:34)
at org.gradle.internal.execution.impl.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:69)
at org.gradle.internal.execution.impl.steps.TimeoutStep.execute(TimeoutStep.java:49)
at org.gradle.internal.execution.impl.steps.CatchExceptionStep.execute(CatchExceptionStep.java:34)
at org.gradle.internal.execution.impl.steps.CreateOutputsStep.execute(CreateOutputsStep.java:49)
at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:42)
at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:28)
at org.gradle.internal.execution.impl.steps.CacheStep.executeWithoutCache(CacheStep.java:133)
at org.gradle.internal.execution.impl.steps.CacheStep.lambda$execute$5(CacheStep.java:83)
at java.util.Optional.orElseGet(Optional.java:267)
at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:82)
at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:37)
at org.gradle.internal.execution.impl.steps.PrepareCachingStep.execute(PrepareCachingStep.java:33)
at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:38)
at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:23)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:95)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.lambda$execute$1(SkipUpToDateStep.java:90)
at java.util.Optional.orElseGet(Optional.java:267)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:90)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:36)
at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:34)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:109)
at org.gradle.api.internal.tasks.execution.ResolveIncrementalChangesTaskExecuter.execute(ResolveIncrementalChangesTaskExecuter.java:84)
at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:91)
at org.gradle.api.internal.tasks.execution.FinishSnapshotTaskInputsBuildOperationTaskExecuter.execute(FinishSnapshotTaskInputsBuildOperationTaskExecuter.java:51)
at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:102)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:74)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:109)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
at org.gradle.api.internal.tasks.execution.StartSnapshotTaskInputsBuildOperationTaskExecuter.execute(StartSnapshotTaskInputsBuildOperationTaskExecuter.java:52)
at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:93)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:45)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:94)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:63)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:49)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:46)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.lang.Thread.run(Thread.java:748)
> Task :sdks:python:test-suites:portable:py37:installGcpTest FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/py37/build.gradle'> line: 60
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 255
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:setupVirtualenv'.
> Failed to create directory '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/1398941891'>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 20s
80 actionable tasks: 59 executed, 21 from cache
Publishing build scan...
https://gradle.com/s/vf6riub6gxjp4
FAILURE: Build failed with an exception.
* What went wrong:
Could not add entry '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/runners/google-cloud-dataflow-java/worker/windmill/build/extracted-include-protos/main/google/api/distribution.proto'> to cache fileHashes.bin (<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/.gradle/5.2.1/fileHashes/fileHashes.bin).>
> java.io.IOException: No space left on device
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 21s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Python37 #1825
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python37/1825/display/redirect?page=changes>
Changes:
[github] Fix a bug in performance test for reading data from BigQuery (#11062)
------------------------------------------
[...truncated 2.64 MB...]
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-8:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-10:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-5:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-7:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Process Process-6:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 652, in runner
keyboardCaught, shouldStop, loaderClass, resultClass, config)
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 700, in __runner
for test_addr, arg in iter(get, 'STOP'):
File "<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py",> line 675, in get
return testQueue.get(timeout=config.multiprocess_timeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
error: [Errno 28] No space left on device
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:portable:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/direct/py37/build.gradle'> line: 60
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:hdfsIntegrationTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:direct:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'> line: 89
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 47s
82 actionable tasks: 60 executed, 22 from cache
A build scan cannot be produced as an error occurred spooling the build data.
Please report this problem via https://gradle.com/scans/help/plugin and include the following via copy/paste:
----------
Gradle version: 5.2.1
Plugin version: 2.3
java.lang.IllegalStateException: java.io.IOException: No space left on device
at com.gradle.scan.a.e.b.flush(SourceFile:233)
at com.gradle.scan.a.e.b.close(SourceFile:244)
at com.gradle.scan.a.e.c.close(SourceFile:66)
at com.gradle.scan.plugin.internal.n.a(SourceFile:78)
at com.gradle.scan.plugin.internal.e.c.c.b(SourceFile:125)
at com.gradle.scan.plugin.internal.e.c.c.a(SourceFile:117)
at com.gradle.scan.plugin.internal.o.a$a.a(SourceFile:30)
at com.gradle.scan.plugin.internal.o.a$a.a(SourceFile:19)
at com.gradle.scan.plugin.internal.o.a.c(SourceFile:60)
Caused by: java.io.IOException: No space left on device
at com.gradle.scan.a.e.b.flush(SourceFile:231)
... 8 more
----------
FAILURE: Build failed with an exception.
* What went wrong:
Failed to notify root build lifecycle listener.
> Could not update /home/jenkins/.gradle/caches/5.2.1/file-changes/last-build.bin
> Could not update <https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/.gradle/5.2.1/fileChanges/last-build.bin>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 47s
FAILURE: Build failed with an exception.
* What went wrong:
Could not receive a message from the daemon.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org